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FOREWORD 


Although  many  Navy  managers  recognize  that  Science  and  Technology  (S&T)  programs 
are  essential  ingredients  of  a  vigorous  research  and  development  (R&D)  organization’s  technical 
capabilities,  few  are  guided  by  this  knowledge  to  take  the  strong  actions  needed  to  support  basic 
science  and  research. 

Over  many  fiscal  years,  Navy  leadership  has  struggled  with  this  dimension  of  the  Navy 
laboratories,  caught  between  the  near-term  need  to  fund  operational  forces  and  sustain  adequate 
force  structure  and  the  competing  need  for  the  reliable,  adequate,  long-term  S&T  investment 
required  to  meet  future  challenges. 

Thus,  managers  of  S&T  programs  are  frequently  asked  to  show  justifying  measures  for 
this  investment.  This  has  always  been  a  challenge. 

The  information  presented  here,  originated  by  the  Dahlgren  Division  S&T  Council,  is  an 
attempt  to  understand  and  specify  a  context  for  the  metrics  of  S&T  as  applied  to  Navy 
laboratories.  There  is  more  to  be  done,  but  it  is  felt  that  the  material  collected  and  the 
conclusions  drawn  to  this  point  should  be  documented  now  and  made  available  to  others. 

Several  S&T  Council  members  participated  in  the  various  phases  of  this  study.  Mr.  Larry 
Triota  (Z08),  the  Dahlgren  Division  S&T  Director,  frequently  let  us  know  his  opinions  and  made 
suggestions.  Ms.  Natalie  Heffeman  (W43)  was  also  helpful  with  her  assenting  and  dissenting 
opinions.  Others  who  contributed  their  thought  through  the  early  phases  of  this  study  were 
Messrs.  Alan  Shimp  (Q06),  Steven  Anderson  (W1 1),  and  Douglas  Marker  (W10). 

In  particular,  the  authors  wish  to  thank  Captain  Joseph  McGettigan,  Commander,  Naval 
Surface  Warfare  Center,  Dahlgren  Division  (NSWCDD),  for  enduring  a  two-hour  session  with 
us  on  this  subject.  Prior  to  the  session,  it  was  suggested  that  this  subject  would  be  difficult  to 
grasp  in  a  few  charts.  It  proved  to  be  more  effectively  understood  when  the  full  weight  of 
researched  material  was  confronted  and  viewed  in  the  larger  context  of  the  historical  evidence. 


Approved  by: 


ROBERT  V.  GATES,  Head 

Strategic  and  Weapon  Control  Systems  Department 
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CHAPTER  I 
INTRODUCTION 


1. 1  PREFACE 

The  subject  of  Science  and  Technology  (S&T)  metrics  has  come  up  frequently  during  the 
early  months  of  2004;  as  the  Naval  Sea  Systems  Command  (NAVSEA)  restructuring  plan  has 
unfolded,  and  Navy  management  has  tried  to  determine  what  it  can  do  to  save  money  to  invest  in 
recapitalizing  the  fleet.  In  this  context,  the  question  arises,  “What  is  the  value  to  the  Navy  of  its 
investment  in  science  and  technology?”  Recently,  this  investment  has  been  about  $2  billion  each 
year. 


This  is  not  a  new  question.  History  books  abound  in  stories  on  this  subject. 

There  is  also  another  question  that  is  asked.  It  is  probably  true  that  in  the  21sl  century 
there  are  no  responsible  politicians  who  would  promote  the  idea  that  S&T  investment  should  be 
done  away  with  all  together.  However,  as  recently  as  the  last  century,  Hitler  was  quoted  as 
saying  to  Max  Planck,  “Our  national  policies  will  not  be  revoked  or  modified,  even  for  scientists. 
If  the  dismissal  of  Jewish  scientists  means  the  annihilation  of  contemporary  German  science, 
then  we  shall  do  without  science  for  a  few  years.” 

Today,  the  question  would  be,  “How  much  S&T  investment  is  enough?”  This  suggests 
that  there  is  an  expected  return  on  this  investment,  and  that  the  return  should  justify  the 
investment.  This  “return”  is  closely  related  to  the  “value”  referred  to  in  the  first  question;  and 
the  “return”  is  sometimes  stated  another  way,  “What  S&T  has  “transitioned”  to  the  warfighter 
lately?” 


To  address  this  problem,  a  subgroup  of  the  Dahlgren  Division  S&T  Council  first  began 
discussing  this  in  May  2004,  thinking  about  S&T  metrics,  specifically  the  return  on  investment 
and  transitions,  and  plotting  a  course  to  an  answer.  The  group  continued  to  meet  about  weekly 
through  December.  Each  meeting,  it  seemed,  led  to  new  insights  into  this  “metric  beast.”  The 
literature  is  filled  with  studies  on  S&T  metrics,  and  a  recent  book,  The  Metrics  of  Science  and 
Technology  was  purchased  and  studied.  * 

A  common  theme  seemed  to  appear  in  the  book  and  many  other  references;  S&T  is 
difficult  to  measure  meaningfully.  By  this,  it  is  meant  that  while  there  are  many  measures  -  like 
publications,  patents,  and  awards  -  most  of  them  are  not  helpful  if  a  measure  of  the  quality  of  the 


’  The  Metrics  of  Science  and  Technology,  by  Eliezer  Geisler,  Quorum  Books,  2000. 
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S&T  is  desired.  Also,  most  of  the  measures  do  not  happen  during  the  research;  but  long  after  it 
is  completed  and  made  public,  which  is  not  very  helpful  in  making  management  decisions. 

The  result  was  that  the  S&T  Council  decided  that  it  was  not  sufficient  to  conclude  that 
this  is  an  impossible  task.  Navy  management  continues  to  ask  the  questions;  and  if  we  do  not 
answer  them,  someone  else  will.  The  following  sections  and  paragraphs  describe  the  results  of 
this  effort. 


NOTE 

The  reader  should  be  aware  of  the  fact  that  this  document  was 
constructed  from  briefing  notes  and  minutes  taken  from  the 
spontaneous  interaction  of  S&T  Council  members  and  others  who 
had  relevant  thoughts  on  this  crucial  subject.  That  said,  this 
document  may  not  appear  in  a  polished  format,  but  the  ideas  that  it 
is  meant  to  convey  are  clear. 


1.2  SUMMARY 

As  a  framework  for  this  study,  it  was  decided  to  describe  the  prevailing  atmospheric 
impressions  and  attitudes  in  the  Navy,  as  we  perceive  them,  that  makes  this  study  necessary. 
Figure  1  contains  the  six  statements  that  can  be  heard  in  discussions  or  even  found  in  papers. 
The  arguments  and  conclusions  provided  in  this  paper  counter  these  inaccurate  and  misguided 
beliefs. 


•We’ll  never  get  anything  out  of  S&T. 

•S&T  is  a  sandbox  operated  for  the  benefit  of  the 
scientists. 

•Level-of-effort  programs  are  inefficient,  stagnant, 
and  non-productive. 

•Industry  will  invent  and  develop  everything  we  need. 
•During  a  war  S&T  can  be  given  a  lower  priority  and 
placed  on  the  back  burner. 

•“The  DOD  tabs,  created  for  the  purpose  of 
technology  transition,  are  widely  judged  to  be 
incapable." _ 


FIGURE  1.  PREVAILING  ATTITUDES 

To  balance  these  prevailing  impressions,  the  team  decided  to  list  what  their  beliefs  were, 
i.e.,  what  we  assert  to  be  true  about  S&T.  Figure  2  presents  those  assertions.  These  assertions 
are  supported  throughout  this  study. 
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1.  In-House  S&T  has  great  value  to  the  Naval  Enterprise. 

2.  The  level  of  S&T  required  to  support  the  Navy  is  independent  of 
the  number  of  ships  or  sailors.  Rather  it  depends  only  on  what 
technologies  are  deployed  and  what  might  be  needed  in  the 
future. 

3*  It  is  dear  that  the  Navy  wants  the  benefits  of  new  S&T.  (■♦.at  a 
“reasonable"  cost) 

4.  Low  hanging  S&T  “fruit”  is  a  myth. 

5.  The  loss  of  institutional  technical  competence  leads  to  failure. 

6.  S&T  projects  never  go  according  to  plan. 


FIGURE  2.  ASSERTIONS 

The  conclusions  that  we  have  reached  were  not  obvious  to  the  team  at  the  study 
inception;  although  as  we  look  back  on  them,  they  are  fundamental  to  S&T  activities  and  should 
have  been  realized  long  ago.  Figure  3  lists  the  six  concluding  statements.  The  first  refers  to  the 
assertions  about  Navy  S&T  that  are  listed  in  Figure  2,  The  second  conclusion  is  a  cautionary 
statement.  It  says  that  management  needs  to  think  carefully  about  how  they  define  their  project 
objectives.  If  the  investigators  are  asked  to  summarize  their  projects  at  year’s  end  by  listing,  for 
example,  the  numbers  of  published  papers  and  patent  applications  their  team  has  accomplished, 
then,  to  be  sure,  there  will  be  papers  published  and  patents  applied  for  -  perhaps  at  the  expense 
of  other  actions  that  may  have  led  to  more  useful  results. 


Conclusions 

*  Metrics  can  supply  evidence  that  the  basic  S&T  assertions  in  this  brief  are  relevant. 

•  S&T  metrics  can  be  defined  and  collected  in  response  to  specific  questions,  but  you  tend 
to  get  what  you  choose  to  measure. 

•  The  immediate  ROI  for  Navy  S&T  is  its  contribution  to  the  quality  and  development  of 
“our  people,  (who)  will  determine  our  future  success”*.,, or  failure, 

*  Transitions  and  speed-of-transition  are  not  significant  measures  of  S&T  performance. 
They  may  be  better  measures  of  how  well  the  entire  RDT&E  acquisition  process  is 
working, 

*  A  better  measure  of  S&T  is  how  well  is  Navy  S&T  addressing  current  and  future  Navy 
needs,  and  how  prepared  is  the  workforce  to  address  those  needs. 

•  The  size  of  the  Navy  S&T  budget  and  in-house  workforce  should  be  determined  by  what 
vou  want  it  to  do,  l.e.  what  Navv  capabilities  need  to  be  enabled. 


FIGURE  3.  STUDY  CONCLUSIONS 

The  third  conclusion  puts  near-term  Return  on  Investment  (ROI)  in  its  proper 
perspective.  It  is  the  workforce  that  benefits  from  hands-on  S&T  projects,  and  the  Navy  benefits 
from  this  experience  and  the  resulting  increased  competence. 

There  are  several  reasons  for  the  fourth  conclusion.  S&T  provides  enabling  technologies, 
but  acquisition  programs  must  provide  funds  for  further  development  and  integration  of  the 
technology  into  their  systems.  S&T  programs  do  not  have  this  kind  of  money.  Therefore,  they 
cannot  control  transitions.  Advanced  Concept  Technology  Demonstrations  (ACTDs),  as  an 
example,  require  the  services  to  commit  funds  before  the  project  proceeds,  and  thus,  in  this  case, 
technologies  are  likely  to  transition.  The  Future  Naval  Capabilities  (FNC)  Program,  as  an 
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opposite  example,  does  not  require  services  to  commit  funds,  just  “promise”  that  they  like  the 
technology  and  if  it  works  they  are  interested.  These  projects  are  frequently  canceled,  because 
an  acquisition  office  loses  interest. 

Rather  than  transitions  as  a  measure  of  the  success  or  failure  of  an  S&T  project,  our  fifth 
conclusion  is  that  a  better  measure  might  be  whether  the  S&T  workforce  is  addressing  Navy 
needs  and  how  prepared  is  the  workforce  to  do  the  work,  Transitions  are  a  better  measure  of 
how  well  the  entire  Research,  Development,  Test,  and  Evaluation  (RDT&E)  acquisition  process 
is  working. 

Finally,  the  sixth  conclusion  gets  to  the  question  how  much  S&T  do  we  need.  This 
should  be  determined  by  what  you  want  the  S&T  workforce  to  do,  e.g.,  provide  enabling 
technology  to  close  identified  Navy  Capability  Gaps. 

During  the  study,  there  were  a  few  statements  that  kept  coming  up  that  seemed  to  have 
the  character  of  truths.  There  are  quite  likely  many  more  that  could  be  listed  here,  but  the  ones 
that  we  chose  to  put  at  the  very  front  of  this  presentation  are  listed  in  Figure  4.  The  first  gets  to 
the  heart  of  the  S&T  metrics  question.  Of  the  multitude  of  measurements  that  can  be  applied  to 
S&T  programs,  an  unbiased  review  by  knowledgeable  experts,  perhaps  twice  a  year,  is  the  one 
that  can  reveal  the  most  about  the  quality  and  appropriateness  of  S&T  projects.  In  addition  to 
providing  insight  to  S&T  management,  peer  observations  relayed  back  to  the  investigators  can 
also  result  in  improvements  to  research  methods  and  procedures. 

•"Peer  Review  is  the  Sacred  Cow  of  S&  T  Metrics ,  n 

*  US&  T  outcomes  are  imagined  and  brought  to  reality  by 
scientists,  engineers  and  technologists.  Yet  these  outcomes  are 
applied ,  misapplied »  and  used/abused  by  leaders  or  managers 
in  government  and  industry  ,  who  do  not  adequately 
understand  them ,  their  potential ,  and  their  power ;  M 

*  "Be  careful  what  you  measure .  What  you  measure  is  what 
you  wilt  get , ** 

•“Technical  competence  requires  technical  tasking.  ** 


FIGURE  4.  GEMS 

The  second  “Gem”  is  related  to  the  ultimate  success  of  research  -  actual  use  of  the  S&T 
results  by  the  operational  Navy.  It  is  critical  that  the  knowledge  gained  during  the  S&T  phases 
be  carried  forward  to  applications,  to  assure  that  the  technology  is  appropriately  used  by 
acquisition  managers  and,  indeed,  to  ensure  that  the  acquisition  managers  are  informed  of  the 
technology’s  full  potential. 

Third  on  the  list  is  a  restatement  of  our  second  conclusion  already  discussed  above. 

The  final  “Gem”  may  be  the  most  significant  of  all,  and  is  one  that  is  vital  for  a  technical 
workforce.  One  of  the  attitudes  listed  in  Figure  1  is  “Industry  will  invent  and  develop  everything 
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we  need,"  Implied  with  this  is  also  the  need  for  the  government  workforce  to  manage  and 
evaluate  industry’s  products.  Examples  are  numerous  where  this  approach  has  failed,  because 
the  government  team  did  not  have  the  knowledge  and  experience  to  recognize  errors  and 
shortcomings  in  the  Industry  performance.  A  competent  government  technical  workforce  can 
only  be  achieved  by  providing  significant  tasking  and  responsibility  in  similar  disciplines  and 
continuing  that  tasking  and  responsibility  for  multiple  years.  This  results  in  a  technically 
competent  workforce  and  continuity  of  the  knowledge  throughout  the  lifetime  of  a  Navy  system. 
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CHAPTER  2 
OVERVIEW 


2.1  DISCUSSION 

2.1.1  Return  on  S&T  Investment 

The  material  that  follows  is  a  synopsis  of  the  briefing  presented  to  Captain  Joseph 
McGettigan.  It  forms  the  study  as  seen  in  Figure  5.  Retum-on-S&T  Investment  and  Transitions 
are  addressed  in  Paragraphs  2.1.1  and  2.1.2  because  of  their  wide  interest.  Paragraph  2.1.3, 
Assertions,  follows  and  is  covered  by  24  charts  found  in  Appendix  B.  This  is  where  we  attempt 
to  demonstrate  that  our  assertions  have  a  basis  in  fact,  unlike  the  prevailing  impressions  and 
attitudes  mentioned  earlier.  Paragraphs  2.1.4  and  2.1.5  bring  us  back  to  the  “Metrics” 
discussion.  Paragraph  2.1.6  summarizes  the  Conclusions. 


2.1.1  Return  on  S&T  Investment 

2. 1 .2  Technology  Transitions 

2.1.3  Assertions:  These  are  what  we  believe,  and  attempt  to  demonstrate  with 
argument,  historical  examples  and  data. 

2.1.4  S&T  Metrics:  Should  be  collected  in  response  to  specific  questions 
from  “evaluators.”  Several  questions  are  proposed  that  might  be  of 
interest  at  all  levels  of  management. 

2. 1 .5  Potential  Metrics:  Metric  Data  collection  is  not  a  trivial  enterprise. 

2.1.6  Conclusions 

Appendix  A:  How  Much  Is  Enough?  (Appropriate  size  of  the  S&T  Workforce) 

Appendix  B:  Metrics  for  our  Value  statements. 

Appendix  C:  Historical  Examples  of  Dahlgren  products  of  S&T  that  have 
transitioned. 

Appendix  D:  Examples  of  disruption  in  the  scientific  workforce  leading  to 
failure. 


FIGURE  5.  BRIEFING  OUTLINE 

Appendix  A  is  where  the  question  “How  Much  Is  Enough?”  is  addressed.  This  will  be  a 
controversial  answer,  and  is  not  adequately  tested;  but  it  suggests  that  a  numerical  estimate  might 
indeed  be  feasible.  The  remaining  appendixes  are  self-explanatory. 

The  text  that  follows  provides  some  explanation  for  these  sections  although  not  all  of  the 
graphic  material  will  be  covered.  The  writers  hope  that  you  will  spend  time  with  all  the  material 
in  order  to  appreciate  that  these  ideas  are  not  unique  to  this  study.  They  have  been  stated  many 
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times  by  knowledgeable  people  for  many  years.  We  hope  having  this  all  together  in  one  place 
will  increase  its  credibility  and  impact. 

Return  on  the  Navy’s  investment  in  S&T  is  one  of  the  two  metrics  most  frequently 
requested.  Our  investigation  showed  that  ROI  for  S&T  could  not  be  quantified  in  fiscal  terms  in 
general  because  of  the  long  delay  between  the  S&T  effort  and  when  the  technology  is  actually 
incorporated  into  a  Navy  system,  a  period  that  can  be  decades.  Also,  the  performing  Navy 
laboratory  does  not  receive  the  financial  benefit,  as  would  be  the  industry  case. 

A  note  of  caution  is  called  for  here.  In  answer  to  questions  about  what  has  the  Navy 
gotten  from  its  S&T  investment,  it  is  sometimes  tempting  to  make  a  list  on  a  view-graph  and 
show  it  to  a  room  filled  with  people  with  no  connection  to  the  history  of  S&T  contributions. 
Regardless  of  what  is  on  the  list,  someone  will  surely  ask,  and  many  will  think,  “Is  that  all  we  get 
for  $2  billion  dollars  every  year?”  Although  we  are  forced  to  make  these  lists  from  time  to  time, 
remember  the  following. 


CAUTION 

To  show  examples  trivializes  the  results  of  S&T  investment.  It  is 
too  tempting  to  equate  a  few  bullets  on  a  chart  with  the  Navy’s 
$2B  annual  investment. 

The  ROI  most  readily  described,  though  difficult  to  quantify,  is  the  resulting  state  of 
readiness  of  the  technical  workforce  to  respond  to  recognized  Capability  Gaps,  and  to  enable  risk 
reduction  throughout  the  acquisition  process.  S&T  tasking  certainly  contributes  to  a  prepared 
and  experienced  workforce  to  support  in-house  technical  authority  through  continuity  of  core 
technical  competencies. 

Figure  6  is  the  summary  for  the  ROI  section  of  the  report.  Many  past  and  current  military 
capabilities  can  be  linked  directly  to  Navy  investment  in  S&T.  This  investment  must  be 
continued.  It  is  no  exaggeration  when  we  say:  No  Invest  men!  -  No  Starships! 


It  is  widely  accepted  that  ROI,  and  metrics  in  general,  are 
difficult  to  define  for  S&T. 

Many  Military  capabilities  can  be  linked  directly  to  S&T 
discoveries. 

Many  kinds  of  Risk  are  reduced,  including  the  risk  of 
technological  surprise 

A  competent  workforce  is  the  Near-Term  Navy  ROI  for  S&T. 
No  Investments  -  No  Starships! 


FIGURE  6.  ROI  SUMMARY 


7 


NS  WCDD/M  P-06/55 


One  source,  referring  to  the  end  of  the  Cold  War,  cited  U.S.  S&T  capability  as  “an 
additional  form  of  deterrent  against  our  adversaries  ”  This  ROI  would  be  impossible  to  quantify, 
but  certainly  stands  up  to  close  scrutiny. 


2.1.2  Technology  Transitions 

The  second  S&T  metric  frequently  asked  for  is  Technology  Transitions.  Within  the  S&T 
community,  a  transition  is  usually  thought  of  as  any  movement  between  the  three  categories,  6.1, 
6.2,  or  6.3.  (Particularly  if  the  objective  or  metric  being  measured  are  transitions.)  However,  the 
transitions  that  count,  the  ones  usually  sought  when  this  question  is  asked,  are  the  cases  where  a 
technology  is  taken  into  an  acquisition  program  that  eventually  ends  up  as  a  Navy  capability,  and 
this  may  be  a  transition  from  6.2  or  6.3  to  acquisition. 

The  missing  ingredient  in  this  discussion  is  the  money.  While  S&T  provides  a 
“technology  menu,”  the  acquisition  programs  must  provide  the  funding  for  developing  the 
technology  into  an  operational  system.  The  S&T  community  does  not  have  this. kind  of  money 
and,  therefore,  cannot  control  transitions!  The  missing  link  is  the  gap  between  the  two.  A 
“Technology  Transition  Manager,”  focusing  totally  on  bringing  technologies  to  acquisition 
programs,  would  be  the  owner  of  the  “Transition  metric.”  This  position  does  not  currently  exist. 

These  facts  are  known  within  the  Department  of  Defense,  as  evidenced  by  a  multitude  of 
programs  specifically  designed  to  provide  funding  for  transition  of  acquisition-ready 
technologies.  One  example  of  this  is  the  ACTD  Program.  There  are  many  others. 

We  conclude  that  transitions  are  not  a  significant  metric  for  S&T.  A  better  measure  is  the 
degree  to  which  the  S&T  investment  is  focused  on  current  Navy  needs  and  anticipated  future 
needs.  A  corollary  to  this  is  the  preparedness  of  the  workforce  to  address  those  needs.  Figure  7 
summarizes  our  conclusions  on  S&T  Transitions  as  a  metric. 


•ONR,  ASIN,  and  OSD  recognize  that  bridging  the  “Valley  of 
Death”  between  S&T  and  Acquisition  requires  programs  and 
people  dedicated  to  enabling  Technology  Transition. 

S&T  Metrics  -  Quality  and  Relevance 

Acquisition  Metrics:  Budget,  Schedule,  and  Performance 

Transition  Program  Metrics:  Successful  Technology 

Transitions 

-Transitions  and  speed-of-transltion  are  not  significant  measures 
of  S&T  performance.  They  may  be  better  measurers  of  how  well 
the  entire  RDT&E  acquisition  process  is  working. 

-A  better  measure  of  S&T  is  how  well  is  Navy  S&T  addressing 
current  and  future  Navy  needs,  and  how  prepared  is  the 
workforce  to  address  those  needs. 


FIGURE  7.  CONCLUSIONS  ON  S&T  TRANSITIONS  AS  A  METRIC 
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2.1.3  Assertions 

With  ROI  and  1  ransitions  behind  us,  and  still  seeking  to  counter  the  prevailing 
impressions  and  attitudes,  the  team  decided  to  list  its  own  impressions  in  the  form  of  assertions, 
or  beliefs.  Figure  8  lists  our  six  assertions.  This  section  of  the  report  examines  and  supports  the 
validity  of  these  beliefs. 


1.  In-House  S&T  has  great  value  to  the  Naval  Enterprise. 

2.  The  level  of  S&T  required  to  support  the  Navy  is  independent  of 
the  number  of  ships  or  sailors.  Rather  it  depends  only  on  what 
technologies  are  deployed  and  what  might  be  needed  in  the 
future. 

3.  It  is  clear  that  the  Navy  wants  the  benefits  of  new  S&T.  (...at  a 
“reasonable”  cost) 

4.  Low  hanging  S&T  “fruit”  is  a  myth. 

5.  The  loss  of  institutional  technical  competence  leads  to  failure. 

6.  S&T  projects  never  go  according  to  plan. 


FIGURE  8.  ASSERTIONS  -  WHAT  WE  BELIEVE 

This  is  expanded  in  the  list  that  follows. 

A.  We  believe  that  the  Naval  Enterprise  benefits  from  a  robust  S&T  program  through 
vision  to  predict  future  naval  needs;  through  many  dimensions  of  risk  reduction, 
including  the  risk  of  technological  surprise. 

B.  We  believe  that  the  level  of  S&T  needed  to  support  the  Navy  is  independent  of  the 
number  of  ships  and  sailors,  only  depending  on  the  scope  of  technology  currently 
deployed  and  that  may  be  needed  in  the  future,  i.e.,  if  the  fleet  is  reduced  10%  the 
S&T  workforce  should  not  be  correspondingly  reduced.  Only  production  levels 
depend  on  the  size  of  the  naval  force.  The  challenge  here  is  to  appropriately  predict 
the  future  needs,  a  task  requiring  knowledgeable  people  with  vision. 

C.  We  believe  the  Navy  wants  (and  needs)  the  benefits  of  new  S&T.  This  is 
demonstrated  by  numerous  Navy  management  actions  of  recent  years,  e.g., 
establishment  of  the  Naval  Air  Systems  Command  (NAVAIR)  Advanced  Technology 
Investment  Boards,  SUBTECH,  SURFTECH,  CARTECH,  Aegis  Technology 
Review  Board,  and  more.  Admiral  Vem  Clark  has  said  “We  need  a  lot  more  Silver 
Foxes,  ’  referring  to  a  small  Unmanned  Aerial  Vehicle  (UAV)  produced  and  delivered 
quickly  to  our  special  operations  forces  in  the  Persian  Gulf  area.  The  National 
Military  Strategy  -  2004  recognizes  the  importance  of  recapitalizing  and  modernizing 
some  force  elements  “ while  investing  in  programs  that  extend  US  military 
advantages  into  the  future/*  This  desired  future  advantage  is  built  on  the  back  of 
patient,  long-term  investment  in  S&T. 
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D.  We  believe  that  there  are  no  “overnight”  successes,  i.e.,  low  hanging  S&T  “fruit,”  just 
waiting  to  be  picked,  is  a  myth.  The  Silver  Fox,  mentioned  earlier,  has  been 
headlined  as  a  great  S&T  success  story;  because  it  was  delivered  to  Special 
Operations  forces  in  only  60  days.  It  is  never  mentioned  that  research  on  UAVs  that 
led  to  today’s  unmanned  capabilities  began  on  the  top  of  a  Marmon  automobile  in 
1918.  The  thermobaric  bomb  used  in  Afghanistan  in  2002,  touted  as  delivered  in  less 
than  6  months,  was  the  result  of  over  30  years  of  research  in  basic  explosive 
chemistry.  Even  the  many  stories  of  quick  solutions  to  problems  encountered  by 
soldiers  in  the  field  were  only  possible  because  there  was  a  trained,  competent,  and 
experienced  workforce  somewhere  that  understood  the  conditions  and  was  able  to 
formulate  a  solution. 

E.  We  believe,  and  it  seems  obvious  in  the  Navy  context,  that  the  loss  of  institutional 
competence  leads  to  failure,  There  are  many  stories  that  support  this  assertion.  Many 
respected  leaders  have  recognized  this,  but  it  continues  to  occur.  A  well-supported 
S&T  capability  is  a  key  step  on  the  road  to  maintaining  this  technical  competence  in 
the  workforce.  A  recent  example  of  loss  of  technical  in-house  competence  is  the 
Space  Shuttle  disaster.  The  leader  of  the  Mission  Management  Team  at  Johnson 
Space  Flight  Center  told  reporters  after  the  accident  "  We  must  rely  on  our  contractor 
workforce  who  had  the  systems  expertise  to  go  off  and  do  that  analysis..  We  don  7 
have  the  tools  to  do  that.  We  don't  have  the  knowledge  to  do  that  or  the 
background  or  expertise  to  do  that  kind  of  thing."  For  budgetary  reasons,  NASA 
had  contracted  out  its  in-house  technical  capability.  Jim  Colvard,  who  has  written 
and  spoken  frequently  on  this  theme,  said  “ Further ,  military  preparedness  is  a 
continuous  function.  The  retained  intellectual  residuals  from  investment  in  the 
Navy’s  science  and  technology  infrastructure  are  available  on  demand  to  the  Navy. 
Knowledge  and  experience  gained  through  a  contract  operation  may  well  be  lost 
when  the  contract  ends  or  goes  to  another  contractor.  It  is  appropriate  within  our 
free  enterprise  system  that  the  continuous  function  is  in-house  and  the 
discontinuous  one  is  in  industry.  It  takes  a  well-thought  association  of  the  two  both 
to  decide  on  and  to  provide  material  the  Navy  needs."  To  finish  this  discussion, 
another  quote  from  Mr.  Colvard  is  appropriate  “ Coupled  with  downsizing,  the 
mergers  within  the  defense  industrial  base  have  eliminated  the  competitive  market 
in  defense.  This  requires  the  Navy  to  be  an  even  more  technically  competent  buyer. 
Private  industry  is  in  no  position  to  define  missions,  analyze  requirements,  and 
maintain  the  Navy's  technical  safety  net.  If  the  Navy  loses  its  effective  internal 
science  and  technology  structure,  just  when  it  needs  it  most,  it  never  will  regain  it, 
short  of  a  catastrophic  normative  reset." 

F.  We  believe  that  S&T  projects  never  go  according  to  plan.  This  is  less  a  belief  than  a 
fact  of  life.  Secretary  of  the  Navy  James  Forres tal,  in  July  1943,  recognized  this 
when  he  stated  " Here  the  cause  of  research  is  close  to  the  hearts  of  man.  It  is 
staffed  by  civilian  scientists  and  officers  of  the  Navy.  Those  men  know,  far  better 
than  any  of  us,  that  the  results  of  research  cannot  be  placed  on  a  time  schedule, 
that  the  ideas  of  men  of  genius  must  have  time  and  leisure  in  which  to  flower. 
They  know  far  better  than  any  of  us  that  the  dollars  invested  in  research  in  times  of 
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peace  may  mean  the  life  of  the  nation  when  it  goes  to  war.”  A  current  example  of 
this  is  the  Navy's  Future  Naval  Capabilities  Program.  In  this  case,  a  very  high  level 
of  initial  planning  occurred  but  the  transitions  -  the  single  purpose  of  the  FNC 
structure  —  have  been  no  better  than  with  the  former  program  structure,  and 
significant  resources  have  been  wasted  in  the  stop/start  “chum  caused  by  frequent 
project  terminations.  There  are  good  reasons  for  this:  (1)  it  is  impossible  to  predict 
when  a  solution  will  occur;  (2)  the  Program  Manager  has  to  be  convinced  that  the 
technology  will  solve  his  problem;  (3)  on  his  schedule;  and  (4)  when  he  has  the 
dollars  available  and  the  priority  is  there,  to  apply  the  transition  dollars.  Acquisition 
and  operational  managers  normally  focus  on  current  issues,  not  what  the  future  will 
need. 


2.1.4  S&T  Metrics 

Collecting  metrics  for  S&T  can  be  a  costly,  time-consuming  activity,  particularly  if  the 
resulting  data  is  not  useful  to  someone.  This  is  because  there  are  a  host  of  potential  aspects  that 
can  be  measured  and  analyzed.  The  book.  The  Metrics  of  Science  and  Technology '  makes  the 
point  that  metrics  should  be  chosen  only  in  response  to  specific  questions  from  the  evaluator. 
The  presumption  is  that  the  answer  is  important  to  the  evaluator.  With  this  in  mind,  a  set  of 
questions  were  formulated,  as  examples  that  might  be  asked  by  various  levels  of  Navy 
management.  Figure  9  lists  these  questions. 


CNO: 

What  is  the  benefit  to  the  Navy  from  $2B  annual  TOA  in  Navy  S&T  accounts? 

What  is  the  benefit  to  the  Navy  from  the  government  and  contractor  workyears 
devoted  to  management,  identification,  or  development  of  S&T  products? 

CNR: 

How  well  is  my  S&T  money  addressing  Navy  Technical  Needs? 

W  hat  is  the  quality  of  Navy  S&T  compared  to  all  other  S&T? 

NAVSEA: 

How  do  S&T  people  and  funding  support  the  NAVSEA  Technical  Authority 
responsibilities? 

NSWC; 

How  dues  S&T  help  meet  PAD  current  and  future  responsibilities? 

DD  CO: 

Are  my  S&T  demographics  in  line  with  PAD  and  Technical  Pyramid  needs? 

VVhat  is  the  health  of  the  Dahlgren  Division  S&T  program? 

DD  S&T  Director:  Do  mv  S&T  projects  address  PAD  and  other  DoD  need*? 

FIGURE  9.  POSSIBLE  EVALUATOR  QUESTIONS 


1  The  Metrics  of  Science  and  Technology,  by  Eliezer  Geisler,  Quorum  Books,  2000, 
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The  nature  of  the  questions  changes  as  you  get  further  from  the  actual  performing 
investigators.  The  Chief  of  Naval  Operations  (CNO)  wants  to  know  what  benefit  the  Navy  is 
getting  from  their  investment.  The  S&T  Director  at  Dahlgren  Division,  who  manages  the 
Division  investments,  is  more  concerned  with  the  quality  and  appropriateness  of  his  project 
selections.  As  an  example  exercise,  the  Division  Commander’s  question  “What  is  the  health  of 
the  Dahlgren  Division  S&T  program?”  was  taken.  Figure  8  lists  some  program  measurers  that 
might  be  gathered  to  answer  this  question. 


The  color  codes  help  to  categorize  these  16  types  of  metrics  shown  in  Figure  10.  Blue 
indicates  the  ones  that  are  actually  manageable,  under  the  control  of  management.  That  is, 
management  can  take  actions  to  adjust  these  numbers  in  the  short  term.  Only  four  fit  this 
category.  The  Green  color  indicates  those  that  can  be  readily  counted.  For  example,  the  number 
of  publications  resulting  from  the  S&T  projects  can  be  gathered  at  years  end,  or  more  frequently 
if  desired.  It  is  less  possible  to  require  papers  submitted  for  publication  since  it  is  not  possible  to 
schedule  publishable  results.  The  third  color,  Reel,  is  the  problematic  group  of  measures. 
Transitions  may  take  decades  to  achieve,  and  later  yet,  the  ROI  may  not  be  realized  until  the 
technology  is  actually  deployed. 


Health  Indicators 


‘Annual  Budget  -  ONR  Discretionary,  Internal  Discretionary^  and 


Other 

$&  1  Funded  Scientists  and  Engineers 
•Navy  S&T  Budget  Request 
-Navy  S&T  Appropriated 

‘S&E  Demographics  (green \ 

Proposals  submitted  and  endorsed 
■Transitions 
•  Patents 

•C  RADA* 

•Paten I  Disclosures 

•PAD  needs  addressed  ( green J 

•Pillar  alignment  I  green) 

•S&E  atirition  rate-  It  dal  In  discipline,  t 
■Publications,  awards. 


Blue  arc  manageable 
i.c,  within  nur  control, 
(ireen  I  in  media  re 
uuipuJ 

Red  long  term  outcomes 


raining  categon 


•Capabilitv  Cap  Coverage  (green I 

•  ROI 


FIGURE  10,  S&T  HEALTH  METRICS 


Nonetheless,  there  are  measures  that  can  be  gathered  that  give  an  evaluator  a  sense  of  an 
S&T  program’s  value.  Mr.  Fountain2  remarks  “ Presently  there  is  no  widely  accepted  wav  for 
the  federal  government  in  conjunction  with  the  scientific  community  to  make  priority 
decisions  about  the  allocation  of  resources  in  and  across  scientific  disciplines.  While  metrics 


’  ‘“Transforming  Defense  Basic  Research  Strategy",  Augustus  W.  Fountain,  IN,  from  Parameters,  Winter  2004-05, 
Vol.  XXXIV,  no  4.  pages  40  to  54.  Parameters  is  the  US  Army  War  College  Quarterly.  Page  45 
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such  as  the  number  and  quality  of  peer-reviewed  publications,  citations,  graduate  students, 
research  awards,  and  the  level  of  external  funding  are  indicators  of  a  vibrant  research 
program,  they  do  not  necessarily  show  how  the  needs  of  the  warfighter  are  being  met .*  We 
can  help  somewhat  in  this  by  gathering  data  on  the  three  blue  metrics:  Product  Area  Directorate 
(PAD)  Needs  Addressed,  Pillar  Alignment,  and  Capability  Gap  Coverage.  However,  only  at  the 
transition  stage  can  the  resulting  technologies  be  evaluated  (ROI)  for  their  significance  to  the 
warfighter.  Quality  assessment  during  the  S&T  investigation  phase  must  be  left  to  peers:  “ Peer 
Review  is  the  Sacred  Cow  of  S&  T  Metrics  ”  (Geisler). 


2.1.5  Potential  Metrics 

This  section  of  the  material  is  a  listing  of  possible  metrics  associated  with  each  of  the 
Value  Statements  discussed  under  Assertion  1  of  Figure  8  in  Section  2.1.3.  It  is  shown  in 
Figure  1 1.  This  demonstrates  again  that  there  are  many  ways  to  measure  S&T,  but  wariness  is 
called  for  in  their  selection. 


Value  Statements 

1.  In-house  S&T  enables  the  recruiting,  training,  and  retention  of  a 
technically  competent  scientific  and  engineering  workforce. 

2.  Real-time  connections  between  S&T  Workforce  and  acquisition  programs 
enables  focused  technology  for  the  warfighters* 

3.  The  in-house  workforce  is  highly  responsive  because  of  it’s  ability  to 
innovate  and  It’s  knowledge  of  Navy  systems. 

4.  Long  term  continuity  of  experience  is  essential  and  can  be  assured  because 
of  flexibility  -  due  to  the  proximity  of  technical  acquisition  responsibilities  - 
in  the  ability  to  provide  continued  support  for  ihe  technical  experts, 

5.  The  S&T  trained  workforce,  by  its  nature,  is  agile  and  adaptive,  bringing 
vision  and  innovation  to  future  Naval  capabilities. 

6.  Practical  applications  of  new  and  emerging  science  and  technologies  can 
be  focused  to  support  Fleet  needs. 

7.  The  in-house  S&T-  trained  workforce  is  able  to  anticipate  and  provide  the 
capabilities  to  stay  ahead  of  future  threats;  and  reduce  many  dimensions  of 
risk. 


FIGURE  1  l.  VALUE  STATEMENTS  FOR  ASSERTION  #1  -  IN-HOUSE  S&T 
HAS  GREAT  VALUE  TO  THE  NAVAL  ENTERPRISE 


2.1.6  Study  Conclusions 

Our  conclusions  are  listed  in  Figure  12.  They  should  be  supported  by  the  material 
developed  during  this  study. 
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*  Metrics  can  supply  evidence  that  the  basic  S&T  assertions  in  this 
brief  are  relevant, 

*  S&T  metrics  can  be  defined  and  collected  in  response  to  specific 
questions,  but  you  tend  to  get  what  yuu  choose  to  measure, 

*  The  immediate  ROI  for  Navy  S&T  is  its  contribution  to  the  quality 
and  development  of  “our  people,  (who)  will  determine  our  future 
success”*.,, or  failure. 

*  Transitions  and  specd-of-transition  are  not  significant  measures  of 
S&T  performance.  They  tnay  be  better  measures  of  how  well  the 
entire  RDT&E  acquisition  process  is  working. 

*  A  better  measure  of  S&T  is  how  well  is  Navy  S&T  addressing 
current  and  future  Navy  needs,  and  how  prepared  is  the 
workforce  to  address  those  needs. 

4  The  size  of  the  Navy  S&T  budget  and  in-house  workforce  should 
be  determined  by  what  you  want  it  to  do,  i.e,  w  hat  Navy 
capabilities  need  to  be  enabled. 


FIGURE  12.  CONCLUSIONS 


22  THE  REMAINING  QUESTION 

Now  that  we  have  discussed  ROI,  and  the  nature  of  S&T  Transitions,  and  have  suggested 
that  there  are  many  “metrics”  that  can  be  conceived  and  tracked  -  if  one  is  patient  enough  -  there 
is  still  one  burning  question:  “How  Much  is  Enough?  "  From  the  “Big  Navy”  perspective,  is  it 
possible  to  estimate  -  to  a  few  significant  figures  -  the  annual  S&T  investment  that  would 
adequately  provide  a  secure  feeling  that  we  were  ahead  of  any  potential  adversary,  now  and 
particularly  in  the  future,  and  that  would  assure  that  we  produce  Navy  systems,  particularly  the 
big  long-lead  items,  that  are  not  obsolete,  relate  to  the  threats  to  be  encountered,  even  before  they 
are  deployed. 

Over  the  years,  there  have  been  many  observers  who  realize  the  dangers  a  nation  faces  if 
the  annual  investment  in  S&T  is  significantly  disrupted.  This  is  not  obvious  to  many  people, 
including  some  of  our  political  and  military  leaders.  A  few  examples  that  follow  will  illustrate 
this  point: 

A.  During  WW1,  the  American  Chemical  Society  offered  to  supply  chemists  to  the 
Army.  The  Army  replied,  “ Thank  you  very  much,  hut  we  already  have  a 
chemist.”* 

B.  "...  when  (Robert) Millikan  (1930s)  offered  the  services  of  the  National  Research 
Council  to  a  conference  of  generals,  the  assistant  chief  of  staff for  supply  implied 
that  there  was  little  need  for  scientists  to  dream  up  new  weapons  or  to  suggest  ways 
to  use  scientific  breakthroughs-  those  matters  would  be  handled  internally  by  the 
armed  services.”  (Laboratory  Warriors  -  pg  26) 

C.  "Our  national  policies  will  not  be  revoked  or  modified,  even  for  scientists.  If  the 
dismissal  of  Jewish  scientists  means  the  annihilation  of  contemporary  German 
science,  then  we  shall  do  without  science  for  a  few  years.”  (Jungt,  43 ).{ Hitler  to  Max 
Planck) 
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D.  "  The  DoD  should  be  more  ruthless  about  cutting  defense  labs.  There  is  little  these 
labs  offer  that  the  private  sector  can 't  match.  While  some  capabilities  are  unique  to 
the  / DoD] ,  these  are  far  fewer  than  their  proponents  will  admit,  and  many  hark 
back  to  technologies  that  have  long  since  been  bypassed  in  the  private  sector.,.the 
need  for  a  large  defense  laboratory  structure  is  simply  indefensible ..."  Source:  Dov 
Zakheim,  CEO,  SPC  International,  17  Jun  97  HASC  testimony. 

E.  A  quote  from  Laboratory  Warriors ,  page  63:  “f Jpon  taking  office  (1934),  President 
Roosevelt  had  created  a  Scientific  Advisory  Board  under  Karl  T.  Compton,  the 
Princeton  physicist  who  had  become  president  of  MIT,  but  that  board  had  not  been 
able  to  do  much.  It  identified  such  glaring  problems  as  the  Navy’s  inadequate 
salaries  for  civilian  scientists,  which  had  forced  the  navy  to  allow  civilian 
employees  to  patent  their  own  inventions  rather  than  deed  then  to  the  government, 
and  which  had  practically  halted  the  free  cooperation  between  the  navy  and 
commercial  organizations  tike  Belt  labs.  The  Board  recommended  significant 
increases  in  government-supported  military  research:  currently  less  than  2%  of  the 
service's  annual  budgets  were  being  spent  of  R&D.  Roosevelt  brushed  off  that  idea, 
believing  that  during  the  depression  any  new  money  must  be  spent  on  social-relief 
measures 

On  the  other  side: 

A.  In  1940,  President  Roosevelt  was  persuaded  by  Vannevar  Bush  to  establish  a 
National  Defense  Research  Committee  to  improve  American  scientific 
preparedness  for  war.  Bush's  philosophy  for  the  new  organization  was  his  belief 
that  “ know-how ”  could  be  “ acquired  only  by  constant  research"  and  that  the 
quantity  of  research  was  as  important  as  the  quality  in  obtaining  quick  answers 
to  questions  about  such  things  as  the  design  of  new  aircraft.  ** 

B.  '•'•Historical  evidence  proves  there  are  serious  consequences  when  technical 
capability  is  lost  or  technical  advice  ignored (Colvard) 

C.  President  Harry  Truman:  "No  aspect  of  military  preparedness  is  more  important 
than  scientific  research 

For  these  reasons,  a  logical  methodology  for  establishing  the  optimum  level  of  S&T 
effort  might  help  convince  policy  makers  that  they  should  continue  to  support  a  healthy,  robust, 
and  competent  Navy  S&T  workforce.  In  1998,  a  Defense  Science  Board  (DSB)  Task  Force  on 
“Defense  S&T  Base  for  the  21st  Century”  was  asked  to  look  into  the  next  century  and 
recommend  “ How  much  DoD  science  and  technology  (6.1,  6.2,  and  6.3)  is  needed  to  maintain 
continued  U.S.  supremacy  considering  U.S.  and  global  civil  technology ?"  They  surveyed 
industry  to  find  out  what  their  investment  was  and  how  corporate  board  rooms  decided  how 
much  to  invest.  They  reported  that  "No  formula  was  discovered  for  establishing  the  optimum 
level  of  DoD  investment  in  science  and  technology,  but  the  most  successful  industries  invest 
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about  15%  of  sales  in  research  and  development  with  about  5.5%  of  sales  invested  in  research 
(equivalent  to  the  DoD  S&T program).  ” 

This  gives  us  two  bounds  for  the  S&T  activity.:  (1)  Roosevelt’s  Scientific  Advisory 
Board  said  2%  was  not  enough,  while  (2)  the  1998  DSB  Task  Force  said  3.5%  should  be 
adequate.  In  terms  of  the  Navy  2003  Task  Obligation  Authority  (TOA),  during  which  the  Navy 
work  force  was  181,902  civilians,  these  percentages  would  suggest  levels-of-effort  of 

2.0%  =  3638  people:  Not  enough 
3.5%  =  6367  people:  Adequate 

We  do  not  have  the  “real”  number  for  the  Navy  2003  S&T  workforce,  but  Office  of 
Naval  Research  (ONR)  reports  that  there  are  about  4000  Navy  people  working  S&T  at  least  half 
time,  a  number  that  includes  Navy  investigators  working  under  tasking  from  outside  the  Navy. 
This  number  could  be  converted  into  a  level-of-effort  for  comparison  purposes,  but  it  is  clear  that 
it  is  on  the  low  side  of  our  range. 

An  Algorithm:  As  an  exercise,  we  tried  to  calculate  the  level-of-effort  needed  to  address 
the  widely  accepted  Capability  Gaps  produced  by  N7.  These  Capability  Gaps  should  represent  a 
“best  guess”  at  what  the  current  Navy  thinks  it  needs,  but  should  be  adjusted  to  add  in  some 
“future  needs”  that  result  from  a  more  visionary  look  at  the  future.  Our  approach  is  to  estimate 
the  level-of-effort  needed  to  address  fully  this  accepted  set  of  Navy  Needs.  This  would  represent 
a  guess  at  how  much  is  enough,  and  could  be  compared  to  the  boundaries  discussed  above. 

Therefore,  our  S&T  workforce  estimate  will  be  based  on  what  we  want  this  workforce  to 
do,  i.e.,  close  the  Navy  Capability  Gaps. 

An  additional  workforce  dimension  can  be  added  related  to  the  Navy  Technical  Holders 
(or  WHs).  Each  of  these  is  supported  by  a  pyramid  of  “experts,”  some  of  whom  will  be  part  of 
the  S&T  workforce.  (A  partial  survey  of  the  Warrant  Holders  confirmed  that  they  believe  they 
need  this  S&T  component  of  their  support  pyramids.)  Figure  13  is  the  resulting  algorithm. 


1.  S&T  Level  of  Effort : 

Npeople  =  (1  +  Pover)  *  Ns&t 

Ns&t  =  Np/a  *  Na  *  Ng  *  (Ns  +  Ne) 

2-  NAVSEA  Pyramid  Support  : 

Nwhp  =  Nwh  *  Navgwhp  *  Pps&t  *  (1-Poverlap) 

3.  Infrastructure  Cost: 

C  infra  =  n  *  (Npeople  +  Nwhp)  *  WY 

4.  Total  =  N  total  -  l  +  2  is  number  of  people. 

Cost  is  3  +  cost  of  people. 


FIGURE  13.  S&T  WORKFORCE  ESTIMATION  ALGORITHM 
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Notice  that  a  cost  has  been  included  to  cover  infrastructure.  It  should  correctly  be  a 
percentage  of  the  existing  cost  of  laboratories  and  equipment,  but  since  we  do  not  have  this 
number  we  simply  equated  it  to  a  fraction,  n,  of  the  number  of  S&T  people,  priced  at  an  average 
Work  Year  (WY)  cost. 

This  algorithm  is  fully  derived  in  Appendix  A  of  this  paper.  A  few  sample  cases  are 
shown  in  Table  1.  Case  #1  says  that  for  100  gaps,  representing  600  projects,  and  assuming  10% 
overhead,  2640  S&T  work  years  are  needed,  only  about  1.45%  of  the  Navy  TOA  workforce. 
The  case  1  A,  for  another  set  of  parameter  values,  brings  us  up  to  3.46%.  (Note:  this  calculation 
needs  to  be  more  carefully  thought  through  for  appropriate  parameter  values.)  Some  may 
consider  the  first  estimate  conservative,  where  the  second  is  more  in  line  with  the  DSB 
recommendation. 

TABLE  1 .  SAMPLE  WORKFORCE  ESTIMATION  CASES 


Parameters 

Case  1 

Case  1A 

Case  2 

Case  3 

Case  4 

Case  5 

Ng 

100 

100 

100 

75 

100 

75 

Na 

2 

3 

2 

2 

2 

2 

Np/a 

3 

4 

3 

3 

3 

3 

Nproj 

600 

1200 

600 

450 

600 

450 

Ns 

2 

3 

2 

2 

2 

2 

Ne 

2 

2 

3 

2 

2 

3 

Ns&t 

2400 

6000 

3000 

1800 

2400 

2250 

Pover 

0.1 

0,05 

0.1 

0.1 

0.1 

0.1 

n 

1 

1 

1 

1 

1 

1 

WY 

150,000 

160,000 

150000 

150000 

150000 

150000 

P%infra 

0.1 

0.3 

0.1 

0.1 

0.1 

0.1 

Cpeople 

$711,630,000 

$1,209,600,000 

$810,630,000 

$612,630,000 

$756,720,000 

$731,970,000 

Cinfra 

$71,163,000 

$362,880,000 

$81,063,000 

$61,263,000 

$75,672,000 

$73,197,000 

C total 

$782,793,000 

$1,572,480,000 

$891,693,000 

$673,893,000 

$832,392,000 

$805,167,000 

Nwh 

167 

100 

167 

167 

167 

167 

Navgwhp 

60 

60 

60 

60 

60 

60 

Pps&t 

0.3 

0.3 

0.3 

0.3 

0.3 

0.3 

Poverlap 

0.3 

0.3 

0.3 

0.3 

0.2 

0.2 

Nwhp 

2104 

1260 

2104 

2104 

2405 

2405 

Npeople 

2640 

6300 

3300 

1980 

2640 

2475 

‘percentage 

1 .45% 

3.46% 

1.81% 

1 .09% 

1.45% 

1 .36% 

Ntotal 

4744 

7560 

5404 

4084 

5045 

4880 

‘percentage 

2.61% 

4.16% 

2.97% 

2.25% 

2.77% 

2.68% 

‘Percentages  are  of  the  FY  03  Navy  civilian  workforce  of  181 ,902 
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Another  example  can  be  done  for  the  Navy  In-House  Laboratory  Independent  Research 
(ILIR)  Program.  We  have  data  for  1993  to  2001  for  comparison.  Using  the  Number  of  projects 
funded  each  year,  Figure  14  shows  results  for  ILIR. 

$18,000,000 
$17,000,000 
$16,000,000 
$15,000,000 
§$14,000,000 
$13,000,000 
$12,000,000 
$11,000,000 
$10,000,000 


FIGURE  14.  NAVY  ILIR  ESTIMATE 


ILIR  Estimate 


Clearly,  the  parameters  can  be  adjusted  to  make  the  match  nearly  perfect;  but  the  point  is 
that  the  estimate  is  pretty  good  for  this  level  of  estimation. 
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APPENDIX  A 

HOW  MUCH  OF  A  NAVY  S&T  EFFORT  DO  WE  NEED? 


This  is  the  burning  question  that  seems  to  be  forming  in  Navy  leadership’s  mind.  Money 
is  scarce.  The  Navy  needs  big  things  -  like  ships.  These  are  billion-dollar  items,  and  the  Navy 
S&  I  budget  stands  out  as  about  $2  billion,  and  not  much  return  on  this  investment  in  the  short 
term  (some  would  say). 

The  “short  term”  is  the  new  way  to  say  not  much  is  being  produced  that  our  Navy  can  use 
in  the  next  year  or  so.  So,  if  our  S&T  efforts  cannot  help  our  sailors  and  marines  in  the  current 
war,  then  why  do  we  need  it  at  all? 

But,  suppose  we  all  agree  that  “zero”  is  not  the  right  answer  to  our  question.  This  is 
probably  a  statement  that  almost  everyone  will  sigh  up  to,  as  long  as  their  agreement  goes  no 
further  than  this.  The  critical  question  then  becomes:  “If  not  “zero”,  then  how  much  is 
enough?” 

This  is  the  question  we  want  to  try  to  answer. 

When  we  say  “S&T  effort,”  we  mean  a  group  of  people,  with  an  appropriate  laboratory 
infrastructure,  working  on  ways  to  do  things  and  build  things  that  we  currently  do  not  know  how 
to  do.  Sometimes  it  is  even  about  discovering  things  that  you  did  not  know  could  be  discovered, 
and  did  not  know  you  needed  -  the  unknown  unknown.  This  is  a  critical  thought  to  keep  in  mind 
as  we  proceed  along  our  path  to  an  answer. 

We  begin  with  the  notion  that  the  S&T  team  is  trying  to  solve  some  problems,  or  provide 
the  enablers  that  lead  to  the  solutions.  One  way  to  scope  out  the  magnitude  of  effort  needed  is  to 
know  what  the  navy  believes  is  it’s  biggest  needs,  or  gaps  in  our  desired  capabilities. 
Capabilities  are  much  more  productive  to  think  about  than  technologies.  It  is  often  a  pitfall  of 
these  conversations  with  scientists  and  engineers,  because  this  group  naturally  wants  to  leap  into 
the  solution  (i.e.,  the  technologies)  before  the  problem  is  fully  defined  (the  needed  capability). 
Another  way  to  think  about  this  is  to  define  “What”  we  need  before  we  discuss  “How"  to  get 
there  , 


A.l  CAPABILITY  GAPS 

The  Navy  is  now  focused  on  Capability  Gaps.  N07  has  a  process.  Naval  Capability 
Development  Process  (NCDP),  that  it  uses  to  sort  through  various  need  statements  before 


A-3 


NSWCDD/MP-06/55 


agreeing  to  a  key  group  of  capabilities  for  the  S&T  community  to  work  on.  ONR  is  focused  on 
FNC,  each  of  which  has  underlying  enabling  capabilities,  and  are  usually  related  closely  to  the 
N07  Capability  Gaps.  Joint  Forces  Command  (JFCOM)  last  year  produced  a  set  of  “S&T 
Capability  Issues"  (7  October  2003)  that  are  important  from  their  perspective.  There  are 
probably  more. 

Thus,  if  we  accept  these  lists,  that  have  high-level  Navy  approval,  we  can  establish  the 
S&T  objectives  and,  hopefully,  avoid  controversy  in  this  initial  step  toward  answering  our 
question, 

Table  A- 1  shows  the  number  of  issues  -  Capability  Gaps  -  defined  by  JFCOM,  N07 
(NCDP),  and  ONR’s  FNC  Integrated  Process  Teams  (IPTs)  (POM-06  ). 


TABLE  A-l .  DEFINED  CAPABILITY  GAPS 


Summary  issues  tasks  (Ecs  or  suh  tasks) 


JFCOM 
NDCP 
POM -06 


36  75  -  do  not  match  to  other  two 

26  *  -  seven  unique  from  POM  06 

25  74  -  six  issues  unique  from  NCDP 


If  we  single  count  the  repeating  issues  in  NCDP  and  POM-06,  we  have  19  on  both  lists 
and  7  and  6  unique,  respectively,  leading  to  32  Capability  Gaps.  Just  about  all  of  the  JFCOM 
issues  are  distinct  from  the  other  two.  So,  across  these  three  sources,  we  count  68  separate 
issues. 

The  JFCOM  and  POM-06  lists  also  provide  under  the  big  issues,  several  subtasks  that 
look  like  Capability  Gaps.  NCDP  does  not  define  their  gaps  below  that  level,  but  JFCOM  and 
POM-06  lists  go  another  level  of  definition  down,  showing  75  and  74  subtasks,  respectively. 
This  equates  to  149  subtasks  for  the  68  issues.  Therefore,  our  first  parameter  is  as  follows: 

Ng  =  number  of  Naval  capability  gaps  (A-l) 

Without  further  defining  the  sub-gaps  here,  let  it  be  said  that  they  frequently  read  as  very 
significant  problems,  not  the  kind  of  thing  that  you  would  look  into  the  eyes  of  an  eager  young 
scientist  and  tell  him  to  go  off  and  solve  this  problem  alone,  (e.g.,  Global  Strike) 


A.2  THE  EFFORT 

Because  these  Capability  Gaps  tend  to  be  significant,  it  seems  reasonable  that  there  might 
be  more  than  a  single  approach,  remembering  that  these  goals  have  never  been  reached  before; 
and  there  might  be  reason  to  believe  that  the  state-of-the-art  is  not  yet  sufficiently  mature  to 
support  a  solution.  Also,  there  is  frequently  more  than  one  technology  direction  for  a  given 
problem,  leading  to  different  groups  of  experts.  Our  second  parameter  now  becomes  the 
following: 
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Na  =  number  of  approaches  per  gap  (A-2) 

Each  approach  may  conservatively  result  in  several  projects  in  a  formula  defined  as 
follows: 

Np/a  =  Number  of  projects  for  each  approach  (A-3) 

A. 3  NUMBER  OF  ANNUAL  PROJECTS 

We  can  now  estimate  the  number  of  projects  each  year  for  this  group  of  Capability  Gaps 
by  using  the  following  formula: 

Nproj  =  Np/a  *  Na  *  Ng  (A-4) 

A  .4  STAFFING 

These  projects  are  staffed  with  scientists  and  engineers,  giving  us  two  more  parameters 
as  shown  in  the  following  formulation: 

Ns  -  number  of  scientists  per  project  (A-5) 

Ne  =  number  of  engineers  per  project  (A-6) 

Thus,  the  scientists  and  engineer  staffing  level  for  these  Ng  Capability  Gaps  is  as  follows: 

Ns&t  =  Nproj  *  (Ns  +  Ne) 

=  Np/a  *  Na  *  Ng  *  (Ns  +  Ne)  (A-7) 

And,  to  be  complete,  there  will  be  an  overhead  staff,  secretaries,  and  maybe  technicians, 
of  some  percentage  of  this  total: 

Pover=  fractional  overhead  (A-8) 

A. 5  S&T  STAFFING  COUNT 

The  number  of  people  performing  the  tasks  associated  with  Ng  Capability  Gaps  is  as 
follows: 

Npeople  =  (1+Pover)*  Np/a  *  Na  *  Ng  *  (Ns  +  Nc)  (A-9) 
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A. 6  WHAT  DO  WE  HAVE? 

We  now  have  six  parameters  that  we  can  vary  to  compute  the  number  of  people  needed  to 
work  on  the  projects  represented  by  Ng  Capability  Gaps.  However,  there  are  some  underlying 
realities  to  consider; 

1 .  All  projects  are  treated  the  same.  Basic  science  (6.1)  projects  look  just  like  capability 
demonstration  (6.3)  projects,  which  we  intuitively  know  isn’t  the  case.  But,  this 
calculation  could  be  divided  into  three  parts  to  allow  the  average  parameters  to  be 
different. 

2.  The  gap  count  probably  doesn’t  adequately  cover  projects  not  specifically  associated 
with  a  gap,  particularly  projects  looking  toward  the  future,  beyond  the  “gap  epoc” 
that  we  “know”  about.  That  is,  the  unknown  unknowns  that  might  someday  reveal 
themselves  during  a  research  experiment  or  test  are  not  captured  if  we  insist  on 
rationalizing  the  proposed  “gaps”  in  today’s  environment.  The  gap  count,  as 
approved  by  N07  or  JFCOM  is  going  to  be  conservative. 

(a)  Also,  this  count  should  include  the  significant  sub-gaps  in  the  gap  definitions 
because  experience  shows  that  the  “big  gap”  is  usually  too  large.  Ex.-Global 
Strike  is  a  “big  gap,  but  it  has  several  challenges  listed  within  its  definition  that 
should  be  used  in  the  gap  count. 

3.  The  number  of  approaches  identified  for  each  gap  will  certainly  depend  on  the  scope 
of  the  specific  gap,  some  being  a  lot  broader  than  others.  For  example,  a  gap  that 
requires  sensing  something  and  interpreting  what  is  observed  could  result  in  separate 
approaches  for  separate  types  of  sensors,  of  which  there  could  be  many;  and  then,  the 
interpreting  part  could  be  separate,  requiring  totally  different  kinds  of  analysis, 
computing,  hardware,  staffing,  etc.,  to  cover  the  possibilities.  So,  Na  could  be  2  or 
10.  Each  approach  might  result  in  several  projects.  Thus,  Np/a  could  be  2  or  10. 
This  algorithm  assumes  an  average  across  all  possible  cases. 

4.  The  overhead  adjustment  will  probably  be  a  small  fraction,  say  5  to  10  percent,  e.g., 
Pover  =  0.05.  In  some  cases,  this  might  be  used  to  account  for  other  types  of  people 
than  secretaries  and  an  occasional  manager,  e.g.,  range  technicians,  or  craft-people  to 
build  special  test  apparatus.  But,  for  this  algorithm,  a  single  number  will  be  used, 
probably  between  5%  and  10  %. 

5.  Again,  the  scientist  and  engineer  staffing  will  vary  widely  from  project  to  project. 
The  parameter  values  for  Ns  and  Ne  should  be  carefully  discussed  and  understood  by 
all  who  use  this  algorithm. 

6.  Validation  of  this  algorithm  can  never  be  done  precisely.  But,  it  should  be  attempted. 
Later  in  this  paper  we  will  come  back  to  this. 
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A.7  COST  OF  S&T 

We  now  have  a  number  for  the  workforce  directly  performing  the  S&T  projects 
associated  with  a  set  of  approved  Capability  Gaps.  The  cost  of  this  effort  is  this  people  cost  and 
the  annual  cost  of  the  associated  laboratory  infrastructure.  The  existing  infrastructure  is 
considered  a  one-time  cost,  which  can  be  estimated,  but  will  not  be  a  focus  of  this  paper,  except 
that  the  annual  upkeep  and  improvements  can  be  considered  as  some  fraction  of  the  sunk  cost. 

If  the  sunk  cost  is  known,  then  the  infrastructure  maintenance  might  be  taken  to  be  the 
following: 

Cinfra  =  (P%infra)  *  (sunk  cost)  (A- 1 0) 


where 


P%infra  <  I  (A-l  l) 

Another  way,  perhaps  more  conservative,  is  to  consider  the  infrastructure  sunk  cost  to  be 
a  multiple  of  the  personnel  cost. 

Cinfra  =  n  *  (Npeople  +  Nwhp)  *  WY  (A- 1 2) 


where 


n  >=  1 

and 

WY  =  a  consolidated  work  year  rate. 

and  people  cost  is  as  follows: 

Cpeople  =  Npeople  *  WY 

Then,  the  total  annual  cost  is  figured  as  follows: 

Ctotal  =  infrastructure  +  people 
=  Cinfra  +  Cpeople 


(A- 13) 


(A- 14) 


(A-I5) 


(A- 16) 


A.8  ONE  MORE  DIMENSION 

There  is  one  more  consideration  for  the  S&T  workforce  count  that  might  be  significant. 
NAVSEA  has  a  system  of  technical  experts  acting  as  Warrant  Holders  in  many  specific 
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engineering  areas,  These  Warrant  Holders  are  supported  by  an  identified  underlying  cadre  of 
experts  to  whom  the  Warrant  Holder  can  go  to  seek  advice  when  issues  arise.  It  is  viewed  as 
critical  that  this  cadre,  referred  to  as  the  technical  pyramid  with  the  Warrant  Holder  at  the  top 
(Figure  A-l),  be  constantly  refreshed  in  order  to  ensure  technical  continuity  through  the  life  of 
the  technical  area.  It  is  believed  that  the  S&T  workforce  that  we  are  discussing  provides  an 
important  dimension  to  this  pyramid. 

It  is  known  that  there  are  many  cases  where  S&T  people  move  on  to  acquisition 
programs  with  the  results  of  the  technology  efforts,  Also,  S&T  experience  early  in  a  career  can 
be  found  across  the  warfare  centers  in  development  groups  and  various  levels  of  management. 
This  is  where  two  of  our  assertions  enter  this  discussion: 

A.  The  loss  of  institutional  technical  competence  leads  to  failure. 

B.  In-House  S&T  has  great  value  to  the  Navy  as  an  institution. 


FIGURE  A-l.  TECHNICAL  AUTHORITY  PYRAMID 

The  point  here  is  that  there  is  a  component  to  the  S&T  workforce  that  directly  supports 
the  technical  pyramids.  Any  decision  to  reduce  the  S&T  workforce  should  consider  the 
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consequences  to  maintaining  a  healthy  technical  pyramid  supporting  the  technical  Warrant 
Holders. 

The  question  now  is  how  to  factor  this  into  the  answer  to  the  question  “How  much  S&T 
is  enough?”  One  approach  is  to  survey  the  Warrant  Holders  and  ask  them  how  they  view  this 
component  of  their  supporting  pyramid.  What  do  they  think  is  an  adequate  level  of  this  kind  of 
expertise?  Since  there  are  167Warrant  Holders  in  NAVSEA,  and  some  equivalent  numbers  in 
the  other  system  commands,  this  survey  would  represent  a  substantial  data  collection  effort. 
Maybe  by  estimating  an  average  across  all  of  these  pyramids  we  can  begin  to  see  just  how  many 
scientists  and  engineers  this  might  represent.  If 

Nwh  -  number  of  Warrant  Holders  (A-17) 


and 


Navgwhp 

=  average  number  of  S&T  trained  people  supporting  each  Warrant  Holder  (A- 1 8) 

then  the  S&T  workforce  component  supporting  Warrant  Holders  is  as  follows: 

Nwhp  =  Nwh  *  Navgwhp  *  Pps&t  (A- 19) 

where  Pps&t  is  the  fraction  of  the  pyramid  that  is  S&T  trained. 

This,  however,  needs  to  be  adjusted  for  those  overlapping  between  the  S&T  projects  and  the 
technical  pyramids. 


FIGURE  A-2,  OVERLAP 

If  we  estimate  an  “average”  percentage  of  overlap,  Poverlap,  then  we  can  adjust  our 
number  accordingly: 

Nwhp  -  Nwh  *  Navgwhp  *  Pps&t  *  (1  -  Poverlap)  (A-20) 

If  we  include  these  people  in  our  count,  the  total  S&T  workforce  that  must  be  maintained  is 
formulated  as  follows: 

Ntotal  =  Npeople  +  Nwhp  (A-21) 
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A.9  VALIDATION 

The  validation  of  an  algorithm  like  this  is  critical  to  its  ultimate  credibility.  This  cannot 
be  done  with  great  precision  by  the  nature  of  the  question.  There  will  always  be  controversy 
over  the  selection  of  parameters;  how  to  count  this  or  that;  just  what  the  maximums  or  minimums 
might  be;  or  indeed,  whether  our  question  is  even  answerable.  But  there  are  some  confidence 
building  approaches  that  may  help. 

1.  A  quote  from  “Laboratory  Warriors”,  page  63: 

Upon  taking  office(1934),  President  Roosevelt  had  created  a  Scientific  Advisory 
Board  under  Karl  T.  Compton,  the  Princeton  physicist  who  had  become  president  of 
MIT,  but  that  board  had  not  been  able  to  do  much.  It  identified  such  glaring  problems 
as  the  Navy’s  inadequate  salaries  for  civilian  scientists,  which  had  forced  the  navy  to 
allow  civilian  employees  to  patent  their  own  inventions  rather  than  deed  then  to  the 
government,  and  which  had  practically  halted  the  free  cooperation  between  the  navy 
and  commercial  organizations  like  Bell  labs.  The  Board  recommended  significant 
increases  in  government-supported  military  research:  currently  less  than  2%  of  the 
service’s  annual  budgets  were  being  spent  of  R&D.  Roosevelt  brushed  off  that  idea, 
believing  that  during  the  depression  any  new  money  must  be  spent  on  social-relief 
measures.” 

2.  A.DSB  2001  Summer  Study  says  3%  of  TOA  should  be  S&T.  1  would  take  this  to  be 
an  optimum  number. 

(a)  In  FY=02,  Navy  civilian  workforce  was  181,902.  Then  the  S&T  workforce 
should  have  been  0.03  x  1 8 1,902*  =  5457  people. 

(b)  A  previous  DSB  study,  1998,  stated:  “No  formula  was  discovered  for 
establishing  the  optimum  level  of  DOD  investment  in  science  and  technology,  but 
the  most  successful  industries  invest  about  15%  of  sales  in  research  and 
development  with  about  3.5%  of  sales  invested  in  research  (equivalent  to  the 
DOD  S&T  program).11 

3.  Another  "validation"  approach  would  be  to  agree  on  an  appropriate  in-house/out- 
house  split  for  Navy  S&T  TOA.  Let's  say  40/60  is  "good"  and  reasonable  in- 
house/out-house  distribution.  Then,  using  your  $150k  per  man  year,  my  number 
of  3%  of  civilian  workforce,  yields  150k  x  5457  people  =  S818.6M.  That  is 
818.6/1714.3  (ONR  S&T  TOA)  =  44%  for  in-house  support. 

4.  The  "NLCCG  Community  DON  S&T"  Chart  -  about  number  8  in  the  brief,  suggests 
that  ONR  pays  for  the  salaries  of  approximately  2375  Department  of  the  Navy  (DON) 
employees.  I  arrived  at  this  by  totaling  the  in-house  DON  S&T  for  Warfare  Centers 
and  the  Naval  Research  Laboratory  (NRL)  -$474. 7M  or  S475M  and  using  an 
approximate  man-year  estimate  of  $200k  to  account  for  travel,  purchases  of 
equipment  and  supplies,  etc.  The  2375  is  an  upper  limit.  In  FY03,  Navy  TOA  was 
$1 1  IB,  Navy  S&T  was  approximately  $2B  or  1.8%  of  TOA.  Thus,  if  ONR  funded 


A- 10 


NSWCD  D/M  P-06/5  5 


it's  "fair  share"  of  S&T  costs,  they  would  have  paid  for  approximately  3280  salaries. 
(Robin  Staton) 

5.  The  Public  Sector  Innovation  Working  Group  recommended  1%  of  GDP  for  the 
Department  of  Defense,  National  Science  Foundation,  and  Department  of  Energy.  In 
2004,  GDP  was  $ II, 733.5 B. 

6.  “To  answer  your  question,  "what  the  ONR  S&T  workforce  number  is?"  based  on  the 
attached  report  we  have  been  saying  that  about  4000  people  are  funded  at  50%  or 
more  from  an  S&T  PE.”  (Ernest  L.  McDuffie,  Ph.D.,  Deputy  Director  N-STAR) 

One  of  our  team  members  gave  this  clarification:  (Bob  Kavetski  gave  a  very  good 
brief  at  the  B  department  offsite  today.  He  used  the  "4000  half  time  or  more”  number 
and  I  asked  him  about  it.  That  number  is  their  estimate  of  the  TOTAL  S&T 
workforce  in  the  WCs  +  NRL,  funded  from  ALL  S&T  sources.  He  used  the  chart 
from  NLCCG  (865M  ONR  S&T  to  NLCCG,  475M  spent  in-house).  1  pointed  out 
that  at  200k  per  manyear,  that  is  about  2375  manyears.  He  agrees  that  the  ONR- 
funded  workforce  is  much  less  than  the  4000  number.  He  says  that  NRL  quotes  that 
SI  M  in-house  buys  six  manyears.  Using  that  estimate,  if  the  475M  was  spent  totally 
on  salary  for  S&Es  doing  S&T  work,  ONR  would  be  paying  for  2850  manyears. 
Apparently,  his  estimate  of  4000  relies  on  the  total  NLCCG  Community  S&T  funding 
of  S1428M,  of  which  the  chart  says  $584. 8M  is  spent  in-house.  If  the  584.8M  was 
totally  for  S&T  salaries,  using  the  NRL  estimate  ($167k/manyear),  you  gel  3509 
manyears  of  S&T  expended  in  the  NLCCG.  (Robin  Staton) 

These  six  “observations”  provide  a  ballpark  sense  of  what  the  S&T  workforce  is  today 
and  what  it  perhaps  should  be  if  you  agree  with  the  DSB,  It  should  be  remembered  that  our 
algorithm  provides  a  level-of-effort  estimate,  which  is  different  from  the  people  count  that  Dr. 
McDuffie  refers  to  in  item  6  in  the  above  listing.  Note  that  references  for  these  studies  are  given 
in  reference  listing. 


A.  10  CONTRARY  OPINION 

Here  is  a  contrary  opinion  on  this  approach  that  points  out  the  difficulties  of  trying  to 
forge  an  algorithm  to  compute  the  size  of  the  S&T  workforce. 

“You  have  proposed  an  incredibly  complex  problem.  Some  considerations: 

We  do  not  have  a  science  of  complexity.  We  have  no  idea  what  the  fundamental 
parameters  are  that  define  and  control  complex  systems.  Inventing  technology  and  adapting  it  to 
national  defense  is  a  quintessential^  complex  problem  and  complex  system.  How  do  we  model 
the  individual  components?  How  do  we  know  what  to  define  as  the  individual  "atoms"  of  this 
complex  system?  What  are  the  processes?  How  many  are  there?  Which  ones  are  more 
important  than  others?  How  many  people  does  it  take  to  execute  a  single  process  in  the  set  of 
processes  that  exist  to  discover,  identify,  prove  concept,  inject  into  acquisition  and  support? 
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Which  brings  up  the  issues  of  supporting  the  pyramids.  Not  only  must  we  know  how 
many,  but  the  complexity  of  answering  the  mail  for  each  one.  Is  a  pyramid  going  to  require  a 
single  person  to  keep  smart  on  pumps  or  will  we  require  an  entire  lab  system  to  keep  smart  on 
signal  processing  and  sensor  technology  alternatives?  Will  a  single  POC  for  "questions 
piocessing  suffice  for  several  simple  pyramids  or  will  a  single  leader  with  many  supporting 
infrastructure  folks  be  required  for  a  single  but  complex  pyramid? 

One  cannot  just  look  at  the  number  of  “gaps"  or  problems  to  be  addressed  but  also  the 
relationship  among  gaps,  the  complexity  of  a  gap,  the  severity  or  a  gap,  the  expansiveness  and 
pervasiveness  of  a  gap.  Methods  for  cross-pol ination,  cross-discipline,  cross-mission  solution 
generation  must  be  researched. 

Complex  systems  have  subsets,  which  have  subsets,  which  may  also  be  complex.  There 
are  -goes-intos  and  goes-out-ofs;  there  are  internal  processes,  data  and  resource  streams, 
products/subproducts,  feedbacks,  interconnections  and  interactions  both  positive  and  negative, 
and  some  deltneale-able  border  to  the  system.  However,  for  national  defense  (or  even  the  Navy) 
these  borders  at e  fuzzy.  The  systems  and  subsystems  keep  modifying  as  do  the  processes.  1 
contend  that  we  can  develop  a  science  of  complex  systems  and  define  the  navy/DOD  S&T 
process  form  discovery  to  delivery  to  Life  cycle  support  to  decommission.  But,  we  DO  NOT 
have  that  knowledge  today.  Attempts  to  mathematically  derive  a  number  of  scientist  and 
engineers  that  must  be  involved  in  the  process  (individual  S&T  actors)  without  that  science  in 
place  is  a  speculation,  not  a  numerical  solution. 

Mode!  the  problem  sets,  model  the  solution  processes  in  an  abstracted  idealized  manner, 
model  cross-links  for  synergy,  model  resource  consumption.  Then  we  might  be  able  to  develop 
the  PARAMETERS  that  go  into  your  calculation.  The  algorithms  are  yet  another  story.  I'm  sure 
that  Ba lisle  and  some  others  would  like  the  solution  curve  to  be  a  monotonically  decreasing 
curve  or  perhaps  even  a  step  function  that  rises  rapidly  and  flattens.  BUT,  the  answer  set  is 
much  more  likely  to  generate  an  exponential  curve  that  shows  the  law  of  INCREASING  returns 
without  a  maximum  -—only  a  minimum  below  which  nothing  gets  done  so  why  bother.  Then 
the  question  becomes  how  much  above  the  minimum  should  we  invest  to  do  the  "right"  level. 
But,  the  "right"  level  is  meaningless  except  in  terms  of  national  survival  and  cost  of  our  blood 
and  treasure.  What  metric  says  we  are  doing  "right  "????”  (Larry  Trio  la) 

It  is  true.  This  is  not  an  easy  estimate  to  make.  But,  we  know  that  these  estimates  are 
made  everyday  without  any  data;  and  they  will  continue  to  be  made.  So,  we  want  to  see  if  this 
approach,  identifying  the  objectives  -  the  Capability  Gaps  -  and  approaching  the  question  of 
staffing  needed  to  close  the  gaps,  is  helpful  in  defining  an  appropriate  level  of  S&T  for  the  Navy. 
There  is  historical  precedent  that  S&T  funding  is  determined  often  on  political  grounds  (as 
President  Roosevelt’s  example  shows)  or  on  very  short  term  expectations,  i.e.,  if  we  cannot 
schedule  breakthroughs  and  provide  innovation  to  the  fleet  in  a  year  or  so,  then  the  money  is 
better  used  elsewhere.  Even  today  it  is  still  heard  that  the  money  spent  on  S&T  cannot  produce 
anything  to  support  “this  ”  war,  so  we  should  spend  it  on  something  that  can  -  like  bullets  and 
tanks. 
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A.  1 1  WHAT  ARE  THE  CAPABILITY  GAPS  TODAY? 

Below  are  the  Unclassified  Naval  Gaps  from  the  POM  06  process,  as  provided  from  N07. 
Also,  see  Table  A-2  for  a  list  of  POM-06  prioritized  gaps  and  enabling  capabilities.  Table  A-3  is 
an  example  of  personnel  versus  cost  calculations. 

1 .  Urban/ Asymmetric  Operations 

(a)  Improvised  Explosive  Devices 

(b >  Hostile  Fire  Detection  and  Response 

(c)  Position-Location-Information  in  GPS  Denied  Environment 

(d)  Transparent  Urban  Structures 

(e)  Fortified  Position  Security 
(0  Modular  Scalable  Weapons 

(g)  Defense  of  Harbor  and  Near-Shore  Naval  Infrastructure  Against 

(h)  Asymmetric  Threats. 

2.  Organic  Mine  Counter  Measures- A II  Aspects 

3.  Knowledge  Superiority  and  Assurance 

(a)  Improved  Cooperating/N on -Cooperating  Target  Situational  Awareness 

(b)  Combat  ID-All  Aspects 

(c)  Enhanced  Warfighter  Use  of  IS R 

(d)  Dynamic  Target  Engagement  &  Enhanced  Sensors 

(e)  Unmanned  Combat  Air  Vehicle 

(0  Data  Fusion  to  Identify  Adversary  Intent 

(g)  Optimal  Mix  of  Large  Manned  /  Unmanned  Sensor  Networks 

(h)  Secure  Collaboration  /  Trusted  Processing 

(i)  Advanced  Communications 

(j)  Global  Information  Grid  (GIG)  Compliant  Networking 

(k)  Next-Generation  Command,  Control  &  Decision  Support  Services 

4.  Expeditionary  Logistics 

(a)  Sea  Base  Integrated  Operations 

(b)  Sea  Base  Mobility  and  Interface 

(c)  Sea  Base  Collaborative  Command  and  Control 

5.  Littoral  Anti-Submarine  Warfare 

(a)  Surveillance,  Search,  and  Localization  Sensor  System  Performance 

(b)  Torpedo  Improvements 

(c)  Smart  Use  of  Distributed  Systems  and  Localization  and  Attack  from  UAVs 

(d)  Multi-Torpedo  Salvo  Defense 

6.  Time  Critical  Strike 

(a)  Aircraft  Integra  ted  Self- Protect  ion  Suites 

(b)  Persistent  High  Speed  Strike  Weapon  to  Engage  Time  Critical  Targets 

(c)  Detect  and  Engage  Mov  ing  Targets 

(d)  Discriminate  and  Provide  Terminal  Guidance  for  Weapons  Targeted  at 

(e)  Moving  Targets 

7.  Missile  Defense 

(a)  Advanced  Electronic  Sensor  Systems 

(b)  Long  Range  RF  Detection,  Tracking,  Deception  and  Jamming 


A- 13 


NSWCDD/M  P-06/5  5 


(c)  Sea  Based  Missile  Defense  of  Ships  and  Littoral  Installations 

8.  Fleet/Force  Protection 

(a)  Intent  Determination-  EG/IR  Enhancements,  Proof-0 f-Concept, 

(b)  Non-Lethal  Approach 

(c)  Full  Spectrum  Defense  vs.  Asymmetric  Threats 

(d)  Small  Boat  Detection  With  Proof  Of  Concept  Non-Lethal  Approach 

9.  Littoral  Combat  Power  Projection 

(a)  Advanced  Naval  Fires  Technology 

10.  Advanced  Capabilities  Electric  Systems 

(a)  Battlefield  Power 

(b)  Pulsed  Power 

(c)  Quiet  Drives  and  Ship  Fuel  Cell  Systems 

TABLE  A-2.  POM-06  PRIORITIZED  GAPS  AND  ENABLING  CAPABILITIES 


USMC  &  N3-N5  Gap  1 :  Urban  Counter-Terrorism  [new) 


Urban/ Asymmetric  Ops  EC-1 :  Improvised  Explosive  Devices  Spiral  1 

Urban/Asymmetric  Ops  EC-2:  Hostile  Fire  Detection  and  Response  Spiral  1 

Urban/Asymmetric  Ops  EC-3:  Hostile  Fire  Detection  and  Response  Spiral  2 

Urban/Asymmetric  Ops  EC-4:  Position- Location- Information 

Urban/Asymmetric  Ops  EC-5:  Fortified  Position  Security 

Urban/Asymmetric  Ops  EC-6:  Defense  of  Harbor  and  Near-Shore  Naval  Infrastructure  Aqainst  Asymmetric  Threats 

Urban/Asymmetric  Ops  EC-7:  Improvised  Explosive  Devices  Spiral  2 

Urban/Asymmetric  Ops  EC-8:  Hostile  Fire  Detection  and  Response  Spiral  3 

Urban/Asymmetric  Ops  EC-9;  Transparent  Urban  Structures 

Urban/Asymmetric  Ops  EC-10:  Modular  Scalable  Weapon 

OMCM  Gap  1 :  Capacity  to  Clear  Large  Areas  of  Mines  w/o  Cued  ISR 

OMCM  EC-1A:  MCM  Capacity  Spiral  T 

OMCM  EC-18:  MCM  Capacity  Spiral  2 

OMCM  EC-1C:  MCM  Capacity  Spiral  3 

KSA  Gap  1 :  Joint  Combat  ID 

KSA  EC-1  A  Improved  Cooperating/Non-Cooperating  Target  Situational  Awareness 

KSA  EC-1B  Real-Time  Long  Range  Air  Defense  Combat  ID  in  Support  of  Early  Engagements 

KSA  EC-1C  Combat  ID  information  Management  of  Coordinated  Electronic  Surveillance 

KSA  EC-1  D  Data  Fusion  for  Combat  ID  to  Reveal  Contact  Intent 

KSA  EC-1E  COCOM  to  Marine  Combat  ID, 

EXLOG  Gap  1 :  Strategic  Mobility  to  Move  Forces  Rapidly  to  the  Sea  Base 

EXLOG  EC-1  A  Sea  Base  Integrated  Operations 

EXLOG  EC-1B  Sea  Base  Mobility  and  interfaces 

EXLOG  EC-1C  Sea  base  Interface  Connectors  for  Persistence 

KSA  Gap  7:  Persistent  ISRT  for  Accurate  Target  Discrimination  &  Location 

KSA  EC-7A  Enhance  Warfighter  Use  of  ISR 

KSA  EC-7B  Multi-Source  ISR  to  the  Warfighter 

KSA  EC-7C  Dynamic  Target  Engagement  &  Enhanced  Sensor  Capabilities 

KSA  EC-7D  Automated  Control  of  Large  Sensor  Networks 

KSA  EC-7E  Advanced  Sensors 

KSA  EC-7F  Unmanned  Combat  Air  Vehide 
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KSA  EC-7G  Marine  and  UxV  Tactical  1SR 


KSA  EC-7H  Data  Fusion  to  Identify  Adversary  Intent 

LASW  Gap  1:  Rapid  Sub  Cueing,  Detection  &  Localization  in  Shallow  &  Deep  Water 

LASW  EC-1  A  Surveillance,  Search,  and  Localization  Sensor  System  Performance  and  HWT  improvements 

LASW  EC-1  B  Rapid.  Covert  Surveillance  System  Deployment  and  LWT  Improvement 

LASW  EC-1C  Cross  Field  Processing  and  Smart  Use  of  Distributed  Systems  and  Localization  and  Attack  from  UAVs 

TCS  Gap  2;  Survivability  of  Aircraft  Operating  at  Low  Altitude 

TCS  EC-2A:  Aircraft  Integrated  Self-Protection  Suites 

TCS  EC-2B:  Aircraft  Protection  Suite  Modernization 

MD  Gap  1 :  Sea  Based  Missile  Defense  of  Ships  &  Littoral  Installations 

MD  EC-1A  Advanced  Electronic  Sensor  Systems  for  Missile  Defense 

MD  EC-1  B  Long  Range  RF  Detection  and  Tracking 

MD  EC- 1C  Long  Range  RF  Deception  and  Jamming 

MD  EC-1  D  Over-the-Horizon  Missile  Defense 

MD  EC- IE  Sea  Based  Missile  Defense  of  Ships  and  Littoral  Installations 

KSA  Gap  2;  Optimal  Mix  of  Naval  Sensors  To  Complement  Joint  &  National  Capabilities  to  Meet  Naval  Mission  Reouirements 

KSA  EC-2A:  Optimal  Mix  of  Large  Manned/Unmanned  Sensor  Networks 

KSA  Gap  3:  Computer  Network  Defense  &  Information  Assurance 

KSA  EC-3  A  Secure  Collaboration 

KSA  EC-3B  Trusted  Processing 

TCS  Gap  3:  Persistent  High  Speed  Strike  Weapon  to  Enqage  Time  Critical  Targets 

TCS  EC-3  A;  Persistent  High  Speed  Strike  Weapon  to  Engaqe  Time  Critical  Targets 

TCS  Gap  4:  Weapons  with  Standoff  &  Flre-and-Forget  Capability  against  Moving  Targets 

TCS  EC-4  A:  Detect  and  Engage  Moving  Targets 

TCS  EC-4B:  Discriminate  and  Provide  Terminal  Guidance  for  Weapons  Targeted  at  Moving  Targets 

LASW  Gap  2:  Platform  Defense  against  Undersea  Threats,  Including  Ship  Self-Defense  aqainst  Multi-salvo  Torpedo  Attacks 

LASW  EC-2A:  Two-Torpedo  Salvo  Defense 

LASW  EC-2B:  Four-Torpedo  Salvo  Defense 

KSA  Gap  4:  Ubiquitous,  Secure  Communications  &  Network  Infrastructure 

KSA  EC -4 A  Joint  Maritime  Communications  &  Networking 

KSA  EC-4B  Advanced  Communications  for  FORCE  net 

KSA  EC-4C  GIG-Compliant  Networking 

KSA  EC-4D  GIG  Dynamic  Tacti cat  Networking 

EXLOG  Gap  2:  Sea  Based  Sustainment  CONORS  &  Capacity  for  Persistent  Combat  Operations 

EXLOG  EC-2A:  Sea  Based  Collaborative  Command  and  Control 

EXLOG  EC-2B  Sea  Based  Collaborative  Planning 

OMCM  Gap  2:  Destruction  of  Mines  in  Areas  through  which  Marine  Corps  &  Joint  Forces  must  Maneuver  from  Deep  Water  to  SZ  &  B2 

OMCM  EC-2A;  MCM  for  Maneuver  Spiral  1 

OMCM  EC-2B:  MCM  for  Maneuver  Spiral  2 

OMCM  EC-2C:  MCM  for  Maneuver  Spiral  3 

KSA  Gap  5:  Link  Management  &  Architecture 

KSA  EC-5A  Link  Management  4  Architecture 

KSA  Gap  6:  Common  &  Persistent  Maritime  Picture  on/below  Surface 

KSA  EC-6  A  GIG-ES  UOOP  &  Decision  Making 

KSA  EC-6B  Next-Generation  Command,  Control  &  Decision  Support  Services 

KSA  EC-6C  Trusted  Combat  C2  Capabilities 
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FFP  Gap  2:  Adequate  Detection  &  Engagement  of  Terrorist  &  SOF  Threats  to  Ships  In  port  &  Transitioning 


FFP  EC-2A:  Intent  Determination  -  EO/IR  enhancements,  proof-of-concapt  for  non-lethal  approach 

FFP  EC-2B:  Full  spectrum  defense  vs,  asymmetric  threats 

LCPP  Gap  2:  Naval  Fires  to  Support  Soeed/Oepth  of  Marine  Corps  and  Joint  Maneuver 

LCPP  EC-2A:  Advanced  Nava!  Fires  Technology  Spiral  1 

LCPP  EC-2B:  Advanced  Naval  Fires  Technology  Spira!  2 

FFP  Gap  3:  Counter  Small  Boats 

FFP  EC-3A;  Small  Boat  detection  w  proof  of  concept  non-let  ha  1  approach 

FFP  EC-3B:  Long  range  detection  and  precision  engagement  vs*  coordinated  small  boats 

TOC  Gap  1:  Turbine  Engine 

TOC  EC-1  A  Turbine  Engine:  Reduce  cost  of  Operations  1 

TOC  EC-tB  Turbine  Engine:  Reduce  cost  of  Operations  2 

TOC  Gap  2:  Reduced  Support  Costs 

TOC  EC-2A  Reduce  support  costs  1 

TOC  EC-2B  Reduce  support  costs  2 

ACES  Gap  1 :  Advanced  Pulsed  Power 

ACES  EC-1A:  Quiet  Drives  and  Ship  Fuel  Cell  Systems 

ACES  EC-1  B:  Pulsed  Power 

ACES  EC-1C:  Power  Distribution  and  Control 

ACES  Gap  2:  Battlefield  Power 

IaCES  EC-2A:  Battlefield  Power 

TABLE  A-3.  EXAMPLE  CALCULATIONS 


Parameters 

Case  1 

Case  2 

Case  3 

Case  4 

Case  5 

Case  6 

Ng 

100 

100 

75 

100 

75 

100 

Na 

2 

2 

2 

2 

2 

2 

Np/a 

3 

3 

3 

3 

3 

4 

Nproj 

600 

600 

450 

600 

450 

800 

Ns 

2 

2 

2 

2 

2 

2 

Ne 

2 

3 

2 

2 

3 

2 

Ns&t 

2400 

3000 

1800 

2400 

2250 

3200 

Pover 

0,1 

0.1 

0.1 

0.1 

0.1 

0.05 

n 

1 

1 

1 

1 

1 

1 

WY 

150,000 

150000 

150000 

150000 

150000 

150,000 

P%infra 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

C  people 

$711,630,000 

$810,630,000 

$612,630,000 

$756,720,000 

$731,970,000 

$909,810,000 

Cinfra 

$71,163,000 

$81,063,000 

$61,263,000 

$75,672,000 

$73,197,000 

$90,981,000 

Ctotal 

$782,793,000 

$891,693,000 

$673,893,000 

$832,392,000 

$805,167,000 

$1,000,791,000 

Nwh 

167 

167 

167 

167 

167 

167 

Navgwhp 

60 

60 

60 

60 

60 

60 

Pps&t 

0,3 

0.3 

0.3 

0.3 

0.3 

0.3 

Poverlap 

0,3 

0.3 

0.3 

0.2 

0.2 

0.1 

Nwhp 

2104 

2104 

2104 

2405 

2405 

2705 

N  people 

2640 

3300 

1980 

2640 

2475 

3360 

'percentage 

1.45% 

1,81% 

1 .09% 

1 .45% 

1.36% 

1.85% 

Ntotaf 

4744 

5404 

4084 

5045 

4880 

6065 

•percentage 

2.61% 

2.97% 

2.25% 

2.77% 

2,68% 

3.33% 

•Percentages  are  of  the  FY  03  Navy  civilian  workforce  of  181 ,902 
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Figure  A-3  lists  the  parameters  that  we  need  to  know. 


Ng  =  Number  of  Naval  Capability  Gaps 
Na  =  Number  of  approaches  per  Gap 
Np/a  =  Number  of  projects  per  approach 
Nproj  =  Number  of  annual  projects 
Ns  =  number  of  scientists  per  project 
Ne  =  Number  of  engineers  per  project 
Ns&t  =  S&E  staffing  level  for  Ng  Gaps 
Pover  =  fractional  overhead 

Npeople  =  adjusted  number  of  people  supporting  Ng  Gaps 

n  =  personnel  cost  multiple,  >=  1 

WY  =  consolidated  work  year  rate 

Cpeopie  =  total  people  cost 

Cinfra  =  annual  infrastructure  cost 

Ctotal  =  total  annual  cost  -  infrastructure  plus  people 

Nwh  =  number  of  Warrant  Holders 

Pps&t  =  fraction  of  the  pyramid  people  S&T  trained 

Navgwhp=  average  number  of  people  in  a  technical  pyramid 

Poverlap  =  fractional  overlap  between  pyramids  and  Gaps 

Nwhp  =  number  of  Warrant  Holders  support  people 

Ntotal  =  total  S&T  workforce  people 


FIGURE  A-3.  PARAMETERS  NEEDED 
Figure  14  correlates  to  Figure  A-3. 


A.  12  CONCLUSION 

This  algorithm  (i.e..  Equations  A-9,  A-20,  and  A-2 1 )  represents  an  approach  to  answering 
the  questions  “How  much  Navy  S&T  do  we  need?”  If  not  “zero.”  then  how  much  is 
enough? 

Defining  the  problem,  we  want  to  solve  approaches  the  first  part;  that  is,  what  does  the 
navy  need  that  S&T  can  facilitate  or  enable?  We  have  chosen  the  Capability  Gaps  as  the  answer 
to  this  question  because  they  are  widely  accepted.  Then  the  problem  is  preparing  a  proposal,  so 
to  speak,  for  accomplishing  that  work,  i.e.,  determine  the  staffing  needed  for  the  many  projects 
focused  on  the  Capability  Gaps.  The  process  is  then  to  set  the  parameters  and  compute  the 
answer.  The  issue  is  to  reach  agreement  on  the  parameters. 

Given  that  the  right  people  can  agree  to  a  set  of  parameter  values,  the  next  question, 
finding  the  minimum  workforce  necessary,  is  not  so  easily  accomplished.  The  1 998  DSB  Task 
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Force  presented  an  approach  to  this  minimization  problem  by  hypothesizing  a  capability  function 
and  to  compute  the  workforce.  Then  you  simply  differentiate  the  workforce  — or  something  like 
this.  However,  they  concluded  that  the  capability  function  could  not  be  determined  at  that  time. 

Our  approach,  while  also  struggling  with  some  lack  of  clarity  in  setting  parameters,  at 
least  allows  us  to  discuss  the  variables  and  look  at  sensitivities. 

In  the  end,  we  do  know  something  about  the  answer  -  what  we  do  today,  what  others 
have  felt  was  appropriate,  and  what  our  collective  experience  tells  us. 

So,  when  our  leaders  ask.  How  much  S&T  do  1  need?,  we  can  respond  with  a  set  of 
options  spanning  various  sets  of  desired  capabilities, 

Table  A -4  shows  a  9-year  span  of  funding  versus  output. 

TABLE  A -4.  OUTPUT  VERSUS  FUNDING 


Output 

1993 

1994 

1995 

19% 

1997 

1998 

1999 

2000 

2001 

#"Trcnsitionrf' 

No  Daft 

38 

35 

41 

33 

38 

44 

49 

50 

#  Projects  (~#PIs,  not  FTEs) 

181 

197 

218 

214 

206 

192 

183 

195 

198 

Published  Ftpe^s 

393 

270 

259 

225 

250 

261 

270 

202 

203 

Sbmtted  Ropers 

no 

90 

82 

74 

59 

17 

19 

54 

109 

Books/Chapters 

52 

7 

20 

19 

6 

8 

is: 

16 

8 

FUtents/Patert  Appl  i  cations 

79 

67 

69 

72 

93 

98 

99  105 

94 

Govemrent  Reports 

76 

62 

68 

56 

38 

27 

— 

20 

8 

19 

Di  saltations 

37 

10 

13 

5 

3 

1 

6 

5 

3 

Presentations 

417 

259 

369 

248 

248 

234 

234 

229 

220 

Awmi&'htoncrs 

79 

52 

68 

39 

41 

39 

48 

19 

14 

Finding  (M) 

$15.4 

$16,8 

$17.0 

$15,0 

$13.7 

$13.1 

$13.0 

$14.2 

$14.4 
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APPENDIX  B 

PRESENTATION:  SCIENCE  AND  TECHNOLOGY  METRICS 
AND  OTHER  THORNY  ISSUES 


The  slides  that  follow  comprise  the  presentation  given  to  Captain  Joseph  McGettigan  on 
12  April  2005.  As  shown,  it  has  the  authors’  raw  notes  and  asides  on  individual  slides  that 
provide  insights  into  the  thoughts  that  went  into  creating  presentation.  The  numbering  of  these 
slides  is  formatted  to  match  the  numbering  scheme  of  this  appendix. 

In  addition,  the  Note  Pages,  where  applicable,  are  included  after  the  slide  to  which  they 
are  relevant. 
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v  Prevailing  Impressions  &  Attitudes 

•We’ll  never  get  anything  out  of  S&T. 

•S&T  is  a  sandbox  operated  for  the  benefit  of  the 
scientists. 

•Level-of-effort  programs  are  inefficient,  stagnant, 
and  non-productive. 

•Industry  will  invent  and  develop  everything  we  need. 
•During  a  war  S&T  can  be  given  a  lower  priority  and 
placed  on  the  back  burner. 

•“The  DOD  labs,  created  for  the  purpose  of 
technology  transition,  are  widely  judged  to  be 
incapable.” 

1H  January  iOOJ 

While  the  title  of  this  presentation  addresses  Metrics  of  S&T  first,  we  want  to  begin  with  one  of 
the  Thorny  Issues,  that  is  ,  a  set  of  prevailing  impressions  and  attitudes  about  Navy  laboratories 
that  seems  to  keep  coming  up  around  the  Navy,  the  DOD,  and  even  the  government  in  general. 
This  list  captures  some  of  the  key  (mis)impressions  that  we  will  attempt  to  counter  throughout 
this  material. 
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One  more 


"The  DoD  should  be  more  ruthless  about  cutting  defense  labs.  There  is 
little  these  labs  offer  that  the  private  sector  can't  match.  While  some 
capabilities  are  unique  to  the  |DoD],  these  are  far  fewer  than  their 
proponents  will  admit,  and  many  hark  back  to  technologies  that  have  long 
since  been  bypassed  in  the  private  sector... the  need  for  a  large  defense 
laboratory  structure  is  simply  indefensible..."  Source:  Dov  Zakheim, 
CEO,  SPC  International,  17  Jun  97  HASC  testimony. 


We  could  find  no  evidence  to  support  these  claims 


23  January  2tH» 


And  a  final  unsubstantiated  statement  in  a  public  forum. 
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I.  Return  on  S&T  Investment 


The  ultimate  return  on  S&T  investment  might  be  captured  in  this 
observation,  referring  to  the  end  of  the  cold  war: 

“In  a  way  our  science  and  technology  capability  has 
acted  as  an  additional  form  of  deterrence  against 
our  adversaries.” *  * 


“The  ability  to  learn  faster  than  your  competitors  may  be 
your  only  sustainable  competitive  advantage.  ** 

“Making  predictions  is  tough  -  particularly  about  the  future.” 
Niels  Bohr. 


*  *  “Transforming  Defense  Basic  Research  Strategy”,  Augustus  W.  Fountain,  ill,  from 
Parameters,  Winter  2004-05,  Vol.  XXXIV,  no  4.  pages  40  to  54.  Parameters  is  the  US 
Army  War  College  Quarterly. 

•  **  Defense  S&E  Workforce  Transformation:  The  Way  Ahead.  Statement  by  Arie  de 
Geus,  Royal  Dutch  Shell,  Former  Strategic  Planner. 
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I.  Return  on  investment  =  Results 


our  vetv  recent  examples  of  ROI 

•  Ttfee  already  used  in  the  War  against  Terrorism: 

•  Advanced  Conventional  Munitions  (synthetic  organic  chemistry, 

rheology) 

•  Advanced  Chemical  and  pfologicai  Sensors  (surface  chemistry, 
potymef-chemistry,  surface 'Acoustic  Wave  science,  molecular 

i  ■  biology,  immunology)  A 

’Advanced  Explosives  Detection  (nirffiar  quadrupole 

-  Vgsonance) 

^"One  that  soon  may  be: 

■  Real-Time  Detection  of  Hidden  Pathogens  (RF  science,  signal 

processing) 


I 


BS 


Caution:  To  show  examples  trivializes  the  results  of  S&T  investment.  It 
is  too  tempting  to  equate  a  few  bullets  with  the  $2B  annual  investment. 


2*  January 


Liszka  2002 
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I.  Return  on  Investment 
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I.  Return  on  Investment 

History  of  Military  Critical  Technology  Developments 


_  ,  ,  Hk  t  Natttrv  vi  Xfitthtry  Crtrtv (tl  Technology  Development 

2#  January  JtHJS  * 


Defense  Science  Board 
Comments 
on  Smart  Weapons 
DUSD(S&T)  Smart  Weapons 
Workshop 
4  December  2002 

Arthur  P.  McGregor 
Associate  Director  for  Conventional 
Weapons 

Office  of  the  Deputy  Under  Secretary  of  Defense 
{Science  &  Technology) 
703-588-7406 
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A  competent  workforce  is  the  Near-Term  Navy  ROI  for  S&T. 
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If  they  don’t  know  what  to  ask  for  then  someone  has  to  tell  them  what 
they  need.  This  is  ROI. 


I.  Return  on  S&T  Investment 
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OFFICE  OF  THE  DIRECTOR  OF 

DEFENSE  RESEARCH  AND  ENGINEERING 
3040  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3040 

Feb  16,2005 

Subj:  Research  and  Engineering  Goals 


“There  are  several  Enabling  Technology  Coals  that  may  not  lead  directly  to  systems,  but  are 
vital  for  enhancement  of  military  capability.  The  DoD  should  increase  emphasis  to  advance 
capabilities  in  the  following  enabling  technology  areas: 

1.  Nanotechnology' 

2.  Biotechnology 

3.  Unmanned  and  autonomous  systems 

4.  Quantum  Cormnunications/compuring  technology 

5.  Networked  systems 
6*  Advanced  materials 

Z  Intellectual  capita! ( workforce) 

8.  DoD  R&E  infrastructure 

_ 9*  Modeling,  simulation,  computation,  and  software  for  complex  systems,*1 


Zfi  kmivy  2rntf 


We  include  this  chart  to  show  that  this  idea  that  S&T  is  a  significant  contributor  to  workforce 
development  is  not  revolutionary,  Mr.  Ron  Sega  issued  these  goals  to  the  DoD  and  included  #7, 
“Intellectual  Capital  (workforce)”  as  an  enabling  technology. 


“These  goals  apply  to  all  DoD 
components  conducting 
R&E." 
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FNCs  do  not  work  well  because  there  is  no  actu 
vfavy  commitment  of  acquisition  money. 
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II.  Transitions  -  The  Problem 


•The  Navy  S&T  program  performs  research  and  identifies  ,  develops  and 
demonstrates  technologies.  In  addition,  it  significantly  enables  the 
recruiting,  training,  and  retention  of  a  quality  technical  workforce. 

•True  S&T,  by  its  nature,  does  not  have  a  “guaranteed”  output  or  result, 
and  does  not  deliver  on  predetermined  deadlines. 

•“Failure  to  transition”  is  often  NOT  due  to  poor  S&T  or  irrelevant  S&T. 
•Numerous  ONR,  ASN  and  OSD  leaders  and  programs  have  recognized 
this  situation  and  have  attempted  to  institute  solutions. 

•ONR/OP91:  ATDs,  FNCs,  TcchSolutions 

•ASN:  CTTO*,  Rapid  Technology  Transition  (RTT)  program 

•OSD:  ACTDs,  Tech  Transition  Initiatives,  Quick  Reaction 

Fund... 

•A  specific  example  from  a  recent  OUSD(AS&C)  brief  follows: 


2  X  J  i  rani  ry  2EH1J 


*  CTTO:  Chief  Technology  Transition  Officer 
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DDR&E  and  AS&C  Transition  Programs 

-  Including  Flagship  ACTD  Program 

FY  2004  Current  Estimates 


Prw-Syst*mk  Sytlom*  Acquisition  (Englri**rtnfl  Osvsioprrwril,  Suktkinrrwii  4 

Acqutkltion  DkmOftktrkHon.  LftIP  4  Production)  UainlinancB 


(riitlMl  Prurfuct 
Proem  CAptbibly 


^roduci/PraciSi 


ACTDs 


zp; 


Production  & 
Deployment 


mmi 


Pn&tfuCi'Pr  Dt*kl 

loterliicirr 


Prod  lit  l/Prdcsn 
impftyvsmanl  4  Su*fliimn#rtl 


Jomf  Wa  rfigh  ting  Program  (J  WP)  $  1 QM  | 

P  Tech  Transition  Initiative  SUM 

QRSP  Defense  Acq  Challenge  S1SM 

f  ^QRSP  Quick  Reaction  Fund 


Foreign  Comparative  Testing  £36  Af 

Dual  Use  Science  &  Technology  -  DliS&T  ST0M  j 

Manu/acfurfng  Technology  -  ManTech  5253Af 

r/rfd  Iff  of  the  Defense  Production  Act  S77M 


All  programs  aim  to  place  priority  on  Investing  in  GWOT  solutions  identified 
by  the  Counter  Terrorism  Technology  Task  Force  (CTTTF) 

2H  January  2005 


•  Advanced  Concept  Technology  Demonstration-”What’s  New  with  ACTDs?" 

Ms.  Sue  Payton ,  Deputy  Linder  Secretary  of  Defense,  Advanced  Systems  &  Concepts 
PACOM  Conference  Brief-  10  March  2004 


•  “Tool  Box”  of  Transition  Tools  available. 

•  The  graphic  represent  FY-04  and  FY-05  Defense  Wide  RDT&E  funding.  These  resources 
leverage  Defense  Agency  and  Military  Service  programs,  helping  provide  a  much  quicker 
transition  to  the  warfighter. 

•  The  MANTECH,  Dual  Use,  and  IR&D  lines  are  funded  through  Service  Program 
Elements. 


*  Note  for  AS&C:  FY-04  is  the  “Current  Estimate.”  (adjusted  for  Congressional 
Rescissions,  SBIR  Taxes,  withholds  etc...)  The  FY-05  number  is  the  President’s  Budget 
Submission  .  These  are  the  $$  numbers  the  Hill  is  looking  at 
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IIL  Assertions 

3*  It  is  clear  that  the  Navy  wants  the  benefit 
of  new  S&T. 


This  statement  can  be  supported  by  looking  at  what  the  Navy  is  actually  doing, 

NAVAFR  Advanced  Technology  Review  Boards  (ATRB)  for  Weapons  (the  first,  1 998), 
Strike  Systems,  A  ASW,  Common  Systems,  UAV,  Assault  and  Special  Systems,  and 
Carriers. 

Gun  and  Projectile  Technology  Investment  Board  -  1999 

Office  of  Submarine  Technology  -  10  July  1997 
SUBTECH  -  1997 
SURFTECH  -  July  2002 
CARTECH  -  July  2003 

ASW  Technology  Selection  Process  -  Sep  2004 
AEGIS  Technology  Review  Board  -  late  1 980s. 


Congress  is  also  interested 


“We  need  a  lot  more  Silver  Foxes.” 
Adni.  Vern  Clark.  CNO 


January  JiHIJ 


CARTECH:  Charter  signed  24  July  2003.  The  process  actually  began  in  the  Jan  03 
timeframe.. .but  not  documented  like  the  charter.  Ref.  Gary  Smith,  SPA. 

SURFTEC  dates  from  Betsy  Delong 

SUBTECH  dates  from  a  history  chart  provided  by  Mark  Winters,  SPA. 

Robin  Staton  provided  the  ATRB,  GPTIB,  ASW  Tech,  and  AEGIS  dates. 

CNO  to  the  ONR  sponsored  Naval  Industry  Research  and  Development  Partnership  Conference 
—  Aug  9  2004  -  Defense  News. 


SuWllte  D — Olli*r  Matters 

SffC  1*1  ANNUAL  REPORT  OK  SUBMARINE  TECHNOLOGY 
INSERTION 

(■)  REPORT  REQUIRED  -  (t>  FW  wen  Of  rit«n  jeaW i<K>6, 
ioor  200$,  2«JB.  Ihe  Sec  rotary  d  Ddonse  *MH  fciDmit  io 

rha  tonflfesstenrt  eerTirrutteoi  » report  Ort  lha  submarine 


or  potediily  auLiikafc  to  for  msafoyi 
in  lutmidrww  of  cha  Navy  bo  reduce  the  produdOfi  and 
cqjl>  of  ihe  jubmafme^  ■Hfttis  nainSay  iog  w  rmptfevtna  <hv 
^rtuxil'-  criess  Of  Efio  submarine* 

L?i  The  annual  report  Ipr  a  toal  year  pariafajrfi  1,1  J  shat  be 
sebmiHml  a  I  Lhft  Hiite  Uma  Ihai  I  ha  Prandiiffl  submits  to 
tNj  budge!  (or  that  IWjjJ  ^ar  under  section  1  I0&*l  of  min  j 1 
United  States  Coda 

{b)  CONTENT  —The  nsport  an  submarine  rachnotouiw.  urwter 
Supralon  {Jl  stiaK  Include,  Bpr  «ch  of  Submarines  of  aw 


OJA  Mil  d  the  lochnotogat*  Hi»t  he»*thwmi  rtemanslratQd 
logether  *«h 

:  A]  >i  Dton  tar  by?  iftMflion  d  any  such  tecr»Ote?4»  PuiL  rt*vo 
(lalenronstf  appropriate  loa  Suds  submarines:  and 
,B)  llte  otfnrmtod  owl  d  IMh  technology  msartltmi 
|2)  A  i*4  d  iha  tMJinc^ci,  Hint  * ot  i»an  damomoated. 


1 1  am  i  deny  swii - 

loch.TokoqMes  thal  have  lha  rote  mla>  tar  b#*ng  apfucpnate  to  * 
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III.  Assertions 


4.  Low  hanging  S&T  “fruit”  is  a  myth. 

New  capabilities  do  not  happen  overnight.  There  are 
usually  years  of  step-by-step  progress,  including 
workforce  enlightenment,  that  enable  breakthroughs. 


“Research  on  satellites  and  a  global  positioning  system  began  in  1 946  after  the 
publication  of  an  article  on  geo-stationary  orbits  by  physicist  Arthur  C.  Clarke, 
more  widely  known  for  writing  2001 :  A  Space  Odyssey,  The  first  GPS  satellite 
was  launched  in  1978,  with  the  full  24-$atellite  constellation  completed  on  9 
March  1994"* * “  However,  in  today's  fast-paced  and  dynamic  environment,  the 
Department  of  Defense  cannot  afford  48  years  to  research,  develop,  and  deploy 
critical  technologies  to  the  warfighter.*1  * 


2K  Jjniijry 


*  Fountain,  page  4 1 
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4.  Low  hanging  S&T  “fruit”  is  a  myth. 
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Delivered  in  <  6  months 


III.  Assertions 


Accidents  show  part  of  the  motivation:  you  want  propellants  to  bum  reliably  and  explosives  to  detonate 
violently,  but  only  at  the  right  place  and  the  right  time.  On  29  July  1967,  a  Zuni  rocket  loaded  aboard  an 
A-4  Skyhawk  on  USS  Forrestal’s  flight  deck  cooked  off  and  struck  the  aircraft  to  its  front  (whose  pilot 
was  the  future  Senator  McCain).  134  died  in  the  fires  and  explosions.  USS  Enterprise  and  USS  Nimitz 
later  suffered  similar  accidents.  ONR’s  program  in  synthetic  organic  chemistry  addressed  basic  problems 
underlying  the  accidents.  Energetic  materials  (propellants  and  explosives)  must  meet  conflicting 
performance  demands: 

•  They  must  release  large  amounts  of  energy  rapidly  and  reliably 

•  They  must  be  safe  to  handle-they  must  resist  mechanical  shock,  high  temperatures,  etc, 

•  They  must  be  safe  to  store  for  20  years  or  more 

Energetic  materials  are  difficult  to  characterize.  Most  of  the  work  had  been  empirical — even  Edisonian — 
trial  and  error.  This  means  that  developing  new  explosives  and  propellants  with  desirable  characteristics 
has  been  slow,  expensive,  and  subopt  imal. 

The  challenge  is  to  understand  the  chemical  reactions  sufficiently  to  eliminate  the  trial -and -error 
empiricism  of  development  and  monitoring. 

Explosives  are  composites ,  so  local  effects  cannot  be  ignored,  and  the  chemical  reactions  are  fast.  Basic 
research  has  been  necessary  in:  energetic  crystal  structures,  combustion  mechanisms ,  and  initiation 
mechanisms.  Performers  included  University  of  Chicago,  University  of  New  Orleans,  City  University  of 
New  York,  MIT,  Stevens  Institute  of  Technology,  Cal  Tech,  and  NSWC  Indian  Head.  Among  ONR’s 
performers  in  this  work  was  Ahmed  Zewait  of  Cal  Tech,  whose  development  of  femtochemistry  enabled 
chemists  to  observe  fast  reactions  as  they  occur.  Zewail’s  work  earned  him  the  1999  Nobel  Prize  for 
Chemistry. 

This  basic  research  ultimately  transitioned  to  Naval  Surface  Warfare  Center  Indian  Head's  w'ork  on 
thermobarics.  After  911,  OSD  called  for  thermobarics  (on  19  Sep).  Indian  Head’s  thermobaric  fill  was 
weaponized  and  tested  by  DTRA,  and  used  operationally  against  al  Qaeda  cave  sanctuaries  in  Gardez, 
Afghanistan  on  3  Mar  02. 


B-51 


NSWCDD/MP-06/55 


n 

ffl 


NSWCDD/MP-06/55 


I 

Uj 

GO  CO  O 

«3.c  O 

n"  C 

ll| 

<D  CD  -2 
"O 


o:  j 

3 

co 

i 


W) 

c 

d  • 
H  J+ 

**  'M 

*  B 
C  S 

J  A 

« 

”  *rt 


£ 

..  0 

X  1^, 

r?  ? 

Ph  ^ 

Uh  ^ 

S  Q 

>  £ 
0 
-cs 


CO 


28  January  2005 


NSWCDD/M  P-06/5  5 


Silver  Fox: 
Robot  takes  point 


4.  Low  hanging  S&T  “fruit” 


is  a  myth.  I  m  Assertlons 


(3  o^o 

*  o 


Navy  SBIR  delivers  8  systems  to  NaVy 
Special  Operations  in  60  days.  18  systems 
currently  deployed  to  CENTCOM. 


Initially  fielded  with  I  MEF,  then  transferred  to  SEALs.  ONR  SBIR  performer  Advanced 
Ceramics  delivered  8  systems  in  60  days,  with  ONR  military  operators.  1 8  Silver  Foxes  were 
deployed  within  CENTCOM  in  2003. 

Spiral  development  yielded  quick  operational  roll-outs  and  continuing  improvements.  Rapid 
prototyping  and  insertion  of  technical  innovations  into  subsequent  blocks  have  enabled  the 
system  to  respond  to  emergent  needs.  Its  per  unit  cost  of  $26K  is  approaching  commoditization, 
and  70%  of  that  cost  is  the  EO/avionics  suite. 

Current  endurance  is  3  to  5  hours.  Flight  controls  use  waypoint  navigation  with  a  manual 
override.  The  vehicle  may  be  launched  either  autonomously  from  a  catapult  (see  the  picture  on 
the  left,  above)  or  manually  from  the  ground  or  a  vehicle.  The  system  is  recovered  manually, 
with  autonomous  recovery  coming  in  near  future  blocks.  The  engine  bums  model  airplane  fuel, 
with  future  engines  to  bum  diesel  and  JP-5/8+oil. 

Multiple  UAVs  may  be  controlled  as  a  swarm  trom  a  laptop  computer  with  complete 
Falcon  View  user  interface  and  mission  planning  features.  Sensors  include  EO  cameras  (low-res 
color  and  high-res  low  light  capability)  and  a  FLIR  camera  (320x240  microbolometer).  The 
control  data  link  is  a  line-of-sight  spread  spectrum  system  demonstrated  to  20nm.  The  video 
data  link,  also  line-of-sight,  has  also  been  demonstrated  to  20nm. 
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III.  Assertions 


5.  The  loss  of  institutional  technical  competence 
leads  to  failure. 

The  government’s  primary  requisite  is  to  be  effective  and 
efficient.  In  war  it  is  useless  to  lose  at  half  the  cost  it  would  take  to 
win.  * 

In  1940  President  Roosevelt  was  persuaded  by  Vannevar  Bush 
to  establish  a  National  Defense  Research  Committee  to  improve 
American  scientific  preparedness  for  war.  Bush’s  philosophy 
for  the  new  organization  was  his  belief  that  “know-how”  could 
be  “acquired  only  by  constant  research”  and  that  the  quantity  of 
research  was  as  important  as  the  quality  in  obtaining  quick 
answers  to  questions  about  such  things  as  the  design  of  new 
aircraft.  ** 

2ft  January  200$ 


Colvard  -  “Savings  can  have  a  high  price”.  Nov  1998.  Govexec.com  Magazine. 

**  Laboratory  Warriors,  pg  115.  Vannevar  Bush  was  the  President  of  the  Carnegie  Institution  at 
the  time,  and  was  Chairman  of  the  national  Advisory  Committee  on  Aeronautics. 
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III.  Assertions 


5*  The  Joss  of  institutional  technical  competence  leads  to  failure,  CONT* 


“Historical  evidence  proves  there  are  serious  consequences 
when  technical  capability  is  lost  or  technical  advice 
ignored”.(Colvard) 
e.g.: 

•ValuJet  lost  technical  control  ofits  fleet  and  was  grounded  after  one  of  its  Jets  crashed  in  the 
Honda  Everglades  in  1996 

-  NASA  decided  to  go  through  with  the  doomed  Challenger  launch  in  1986,  despite  technical 
advice  to  delay  it  because  of  cold  weather's  effects  on  the  space  shuttle's  O-rings. 

-The  Navy  lost  its  surface-launched  missile  engineering  capability,  at  least  for  the  short  term, 
in  a  defense  industry  shakeout  that  followed  the  Cold  War. 

VDOT  Experience  -  Too  much  contracted  out,  Not  enough 
experience  remained  on  the  state  team-  led  to  project  failure. 
Commissioner  Shucet  turning  it  around:... ’’ordered  the  hiring  of 
50  licensed  professional  engineers  to  work  in  the  districts.” 


it  Janos™  XW 


Colvard  -  “Savings  can  have  a  high  price.” 


VDOT  -  Aug  2003,  VDOT  Official  Dismissed. 
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HI.  Assertions 


5.  The  loss  of  institutional  technical  competence  leads  to  failure,  CO  NT 


The  recent  NASA  Space  Shuttle  experience.  Too  much  contracted 
out,  the  leader  of  the  mission  management  team  at  Johnson 
Space  Center,  Linda  Ham,  said  in  her  meeting  with  reporters  last 
month  that  she  had  considered  whether  debris  shed  on  liftoff,  now 
believed  to  be  the  fatal  flaw,  could  have  damaged  the  orbiter.  But 
she  said  she  had  relied  on  an  analysis  by  Boeing  that  indicated  no 
threat  to  the  mission  from  the  impact  of  the  foam. 

"We  must  rely  on  our  contractor  work  force  who  had  the  systems 
expertise  to  go  off  and  do  that  analysis,"  she  told  reporters  last 
month.  "We  don't  have  the  tools  to  do  that.  We  don’t  have  the 
knowledge  to  do  that  or  the  background  or  expertise  to  do  that 
kind  of  thing." 

’lUimjiry 


“Shuttle  Inquiry  Uncovers  Flaws...” 
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III.  Assertions 


5.  The  loss  of  institutional  technical  competence  leads  to  failure*  CONT 


“The  Navy  can  never  contract  out  its  ability  to:  understand 
military  problems  in  technical  terms;  know  who  has  the  potential 
to  solve  those  problems;  and  be  able  to  verify  a  correct  solution 
technically  when  it  is  presented.” 

"Further,  military  preparedness  is  a  continuous  function. 

The  retained  intellectual  residuals  from  investment  in  the  NavyTs  science 
and  technology  infrastructure  are  available  on  demand  to  the  Navy.  Knowledge 
and  experience  gained  through  a  contract  operation  may  well  be  lost  when 
the  contract  ends  or  goes  to  another  contractor.  It  is  appropriate  within 
our  free  enterprise  system  that  the  continuous  function  is  in-house  and 
the  discontinuous  one  is  in  industry.  It  takes  a  well-thought  association 
of  the  two  both  to  decide  on  and  to  provide  material  the  Navy  needs.  “ 

21  January  20113 


Colvard  -  “Closing  the  science-sailor  gap” 
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III.  Assertions 


5.  The  loss  of  institutional  technical  competence  leads  to  failure.  CONT 


“Coupled  with  downsizing,  the  mergers  within  the  defense 
industrial  base  have  eliminated  the  competitive  market  in  defense.  This 
requires  the  Navy  to  be  an  even  more  technically  competent  buyer.  Private 
industry  is  in  no  position  to  define  missions,  analyze  requirements,  and 
maintain  the  Navy's  technical  safety  net.  If  the  Navy  loses  its  effective 
internal  science  and  technology  structure  Just  when  it  needs  it  most, 
it  never  will  regain  it,  short  of  a  catastrophic  normative  reset,  “* 


”/  believe  Technical  Authority  is  the  most  important  thing  we  do.  ” 

VADWI  Phillip  M.  Balisle,  COMNAVSEA 


JfiJimury  nm 


*  Colvard  -  “Closing  the  science-sailor  gap”  -  June  2002  -  Naval  Institute  Proceedings. 

Balisle  quote  from  the  briefing  on  NAVSEA  Systems  Engineering  and  Technical  Authority, 
15-16  July  2004, by 


Captain  Steve  Metz 

Assistant  for  Fleet  Naval  Engineering  and  Maintenance  Process  Improvement 

SEA  05N 
(202)  781-3575 
metzsd@navsea.navy.mil 
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.  The  loss  of  institutional  technical  competence  leads  to  failure.  CONT 


“My  goal  is  to  ensure  that  we're  putting  our  resources  where  we  maximize  our  war 
fighting  capability.  My  belief  is  that  the  navy  is  all  about  credible  combat  power  in  the  far 
corners  of  the  earth,  period.  That's  what  this  is  about.” 

Admiral  Vern  Clark  Remarks,  7th  Annual  Navy/CEO  Conference,  November  6,  2001 
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III.  Assertions 

6.  S&T  projects  never  go  according  to  plan. 
Cont. 


Example:  FNC  projects,  where  a  high  level  of  initial  planning 
occurred,  but  the  transitions  have  been  no  better  than  with  the 
former  program  structure,  and  significant  resources  are  wasted 
in  the  stop/start  “chum".* 

Because  (l)  it  is  impossible  to  predict  when  a  solution 
will  occur.  (2)  the  Program  Manager  has  to  be  convinced  that 
the  technology  will  solve  his  problem  (3)  ...on  his  schedule  (4) 
when  he  has  the  dollars  available  and  the  priority  is  there  to 
apply  the  transition  dollars... 

Acquisition  and  operational  managers  normally  focus  on 
current  issues,  not  what  the  future  will  need. 


7*  J  imu  ry  ZfflU 


*  For  this  analysis  see:  “An  Examination  of  the  NAVY’S  Future  Naval  Capability  Technology 
Transition  Process”  by  Robert  E.  McGahem  ,  September  2004.  Naval  PG  School  Thesis. 
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V.  Potential  metrics.  Metric  Data  collection  is  not  a  trivial 
_  enterprise. _ 

Here  are  some  attributes  to  be  considered: _ 

John  H.  Hopps,  Jr.,  Deputy  Director  of  Defense  Research  and  Engineering, 
and  Deputy  Undersecretary  of  Defense  in  the  Department  of  Defense,  has 
stated  that  our  “defense  laboratories  should  have  the  same  attributes  as  our 
transformed  uniformed  military  forces.”  While  the  DOD  is  transforming  to 
build  modular  joint  forces  with  the  attributes  of  speed,  agility,  lethality,  and 
knowledge,  the  service  laboratories  need  to  transform  with  the  parallel 
attributes  of  “productivity;  responsiveness  and  adaptability;  relevance, 
programming,  and  execution;  generation  and  application;  and 
perpetuation  of  knowledge.”  Hopps  argues  that  this  transformation  should 
lead  to  a  greater  investment  in  breakthrough  activities  and  increase  the  reach 
of  the  defense  labs  into  university  basic  research  programs.30  * 


“Peer  Review  is  the  Sacred  Cow  of  S&  T  Metrics.  ”** 


2*  January  20i« 


♦Fountain,  page  47/48 

**  The  Metrics  of  Science  and  Technology”,  Geisler. 
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VI.  Conclusions 


•  Metrics  can  supply  evidence  that  the  basic  S&T  assertions  in  this 
brief  are  relevant 

*  S&T  metrics  can  be  defined  and  collected  in  response  to  specific 
questions,  but  you  tend  to  get  what  you  choose  to  measure. 

*  The  immediate  ROI  for  Navy  S&T  is  its  contribution  to  the  quality 
and  development  of  “our  people,  (who)  will  determine  our  future 
success”*... or  failure. 

*  Transitions  and  speed-of-transition  are  not  significant  measures  of 
S&T  performance.  They  may  be  better  measures  of  how  well  the 
entire  RDT&E  acquisition  process  is  working. 

•  A  better  measure  of  S&T  is  how  well  is  Navy  S&T  addressing 
current  and  future  Navy  needs,  and  how  prepared  is  the 
workforce  to  address  those  needs. 

•  The  size  of  the  Navy  S&T  budget  and  in-house  w  orkforce  should 
be  determined  by  what  you  want  it  to  do,  I.e.  what  Navy 
capabilities  need  to  be  enabled. 


:  H  Jinuiry  SHIS 


*  This  is  from  the  2005  CNO  Guidance  Playbook,  page  3,  Guiding  Principal  #12. 
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Appendix  A:  How  Much  is  Enough? 

S&T  Workforce  Estimation 


During  WW  1  the  American  Chemical  Society  offered  to 
supply  chemists  to  the  Army.  The  Army  replied,  “Thank 
you  very  much,  but  we  already  have  a  chemist”.  * 


“...when  (Robert) Millikan  (1930s)  offered  the  services  of  the  National 
Research  Council  to  a  conference  of  generals,  the  assistant  chief  of  staff  for 
supply  implied  that  there  was  little  need  for  scientists  to  dream  up  new 
weapons  or  to  suggest  ways  to  use  scientific  breakthroughs-  those  matters 
would  be  handled  internally  by  the  armed  services”  (Laboratory  Warriors- 
PS  26) 

Does  this  mindset  still  exist? 


2K  Januirv  2ch)S 


*Jim  Colvard  paper  -  *  “Why  Navy  Laboratories?”,  Jim  Colvard: 
http://www.  nstarweb.com/whynavylab.pdf 

**  “Laboratory  Warriors”,  page  26. 
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Workforce  Algorithm  Appendix  A 


Should  we  separate  6.1,  6.2,  and  6.3? 

How  to  validate  this? 
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Appendix  A 


Validation:  Comparison  numbers 


1 .  “S&T  supports  about  4000  Navy  people  at  least  half  time”.!  1 ) 

2.  DSB  -  1998  Task  Force,  suggests  3.5%  of  TOA  is  reasonable. 
This  would  be  6367  people*. 

3.  The  DSB  2002  study  suggests  3.0%  of  TOA  is  fine. 

This  would  be  5457  people. 

4.  Roosevelt’s  Scientific  Advisory  Board  advised  (circa  1934) 
that  the  then-current  rate  of  investment  was  inadequate:  less 
than  2%  **(2) 

This  would  be  3638  people  today. 

5.  Public  Sector  Innovation  Working  Group  recommended  1%  of  GDP 

for  DoD,  NSF,  and  DOE(3).  2004  GDP  was  $11,733.5  B 


Based  on  2003:  181,902  Navy  civilians) 


2*  JjtuiiY  2IMS5 


(1)  Dr.  McDuffie. 

(2)  **  “Laboratory  Warriors”,  page  63. 


(3)  National  Innovation  Initiative  —  Public  Sector  Innovation  Working  Group  Report  (date 
received  -  8  Mar  2005,  no  date  shown  on  the  report.) 
http://www.compete.org/docs/pdf/NII_Public_Sector_Final%20Report.pdf 
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two-thirds  is  focused  on  evolutionary  Improvements  in  identified  product  needs. 
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yne* 

The  numbers  you  are  referring  to  were  produced  by  Gary  Bess 
mail:  hessgm@navsea - navy  mil;  work  phone;  +1  [202]  731-3441)  Feel  free 
contact  him  with  your  specific  questions  regarding  those  numbers . 


Best  regards, 
--Ernest 


ILIR  data 


Output 

1993 

1994  1995 

1996 

1997 

1998 

1999 

2000 

2001 

#  "Transitions'1 

No  ban 

38j  35 

41 

33 

38 

44 

49 

50 

#  Projects  (-#  Pis,  rot  FTEs) 

181 

197 

218 

214 

206 

192 

183 

195 

198 

Publi^ied  Papers 

393 

270 

259 

225 

250 

261 

270 

202 

203 

Stbmtted  Papers 

110 

90 

82 

74 

59 

17 

19 

54 

109 

Books/Chpters 

52 

7 

20 

19 

6 

8 

13, 

16 

— a 

Patents/ Patent  Applications 

79 

67 

69 

72 

93 

98 

99 

105 

94 

Sever  nmsrrt  Reports 

76 

62 

68 

56 

38 

27 

20 

8 

19 

Dissertations 

37 

10 

13 

5 

3 

i 

6 

5 

3 

Presentations 

417 

259 

369 

248 

248 

234 

234 

229 

220 

Awards/Honors 

79 

52 

68 

39 

41 

39 

48 

19 

14 

Funding  (M) 

$15.4 

$16.8 

$17.0 

$15.0 

$13.7 

$13.1 

$130 

$14.2 

$14.4 

29  tmuirv  Jwtf 


From  Kavetsky  brief  to  B  Dept  Offsite  -  Dec  2004. 


Wayne, 

The  numbers  you  are  referring  to  were  produced  by  Gary  Hess 
(email:  hessgm@navsea.navy.mil;  work  phone:  +1  (202)  781-3441).  Feel  free 
to  contact  him  with  your  specific  questions  regarding  those  numbers. 


Best  regards, 
-Ernest 
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Caution:  To  show  examples  trivializes  the  results  of  S&T  investment.  It 
is  too  tempting  to  equate  a  few  bullets  with  the  $2B  annual  investment. 
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Fischer  and  Michael  Pantazopoulos.  Dr.  Bruce  Copeland  was  the  project  leader.) 
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Early  Dahlgren  Roles  in  the  Development  of  the  Submarine  Launched  Ballistic  Missile  Guidance  System 

Poseidon  Ballistic  Missile 
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!n  less  than  ten  years  DaJilgrcn  had  progressed  from  a  sdf-ini tinted  technical  ad  visory  role  to  become  a  key  member  of  the  SLBM  tire  com rol/gui dance 
team.  The  concepts  developed  lor  POSE  l IX) N  have  continued  in  principle  through  TRIDENTs  I  and  II.  Improvements  in  algorithms,  belter  data,  and  more  computing 
power  in  fire  control  have  allowed  the  full  potential  of  the  concept  lo  emerge. 
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The  composite  navigation  system  (C  NS)  guidance  algorithm  was  developed  in  1999  by  Craig  Phillips  and  Steve  Mai  ye  vac  of  the 
Missile  Systems  Engineering  Branch  (G23)  under  the  Office  of  Naval  Research  Air  and  Surface  Weapon  Technology  Program.  The 
algorithm  was  transferred  to  Draper  Laboratory  in  the  Spring  of 200 1.  Mr.  f-L  C.  Wendt  of  the  Fuze  Branch  (G34)  facilitated  the 
transition  of  this  technology  through  his  role  as  NSWCDD  project  manager  for  the  LCGEU. 
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DoD  S&T — Macro  Scale 
-In  FY06  Constant  Dollars- 


In  FYQ3t  includes  t203M  allocated  to  Def  Emergency  Response  Fund  S&T  in  a  separate  DoD  transfer  account 

12  April  2005 


Mr.  Robert  W.  Baker 
April  19,  2005 

Deputy  Director,  Plans  &  Programs,  ODDR&E 
NDIA  S&T  Conference. 
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DoD  S&T  Programs  as 
Percentage  of  S&T  Budget 


"•-6.1  -•-6.2  -*-6.3 


12  Apnl  2005 


Mr.  Robert  W.  Baker 
April  19,2005 

Deputy  Director,  Plans  &  Programs,  ODDR&E 
NDIA  S&T  Conference. 
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Why  In-House  Technical  Capability? 


(Recurring  Question  -  Recurring  Answers) 


Numerous  studies  have  established  the  value  of  Warfare  Centers  to  technology  transition  and 
innovation.  Mention  highlighted  areas  of  common  findings. 

Most  recently,  the  study  conducted  by  Hugh  Montgomery  for  ONR,  independently  of  our 
study,  resulted  in  virtually  the  same  conclusions. 

The  Hugh  Montgomery  “Naval  S&T  I nvigo ration”  draft  report  references  several  Defense 
Science  Board  reports  which  contain  some  elements  that  support  WCs. 

•  “Efficient  Utilization  of  Defense  Labs”,  draft  Oct  00 

*  Military  labs  represent  an  import  source  of  innovation 

*  “Technology  Capabilities  of  Non-DoD  Providers”,  June  00 

*  DoD  labs  and  centers  should  concentrate  on  unique  military  technologies 
which  are  crucial  to  maintaining  military  preeminence. 

*  Commercial  firms  have  reduced  their  long-term  technology  developments 
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Hugh  Montgomery  brief  |  Fiscal  Year 
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To  begin,  this  is  a  historical  perspective  of  R&D  and  S&T  funding.  Note  the  break  in  the  axis 
and  the  change  in  scale  which  is  necessary  to  even  get  these  curves  to  fit  on  the  same  page. 
Several  trends  are  worth  noting.  First,  the  Navy  R&D  account  tends  to  follow  the  trend  of  the 
Navy  Total  Obligational  Authority  or  TOA,  but,  S&T  does  not.  The  RDT&E  is  consistently 
about  10%  of  TOA.  The  S&T  portion  of  RDT&E  has  stayed  relatively  constant  over  the  recent 
past. 

The  point  here  is  that  S&T,  as  an  investment  for  the  future,  tends  to  remain  stable,  even  when 
TOA  is  drastically  changing;  up  or  down.  Today,  the  Department  of  Navy’s  S&T  account  is 
about  1 .9%  of  TOA;  the  lowest  of  the  three  services. 
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6.  S&T  projects  never  go  according  to  plan. 
Cont. 


“Without  some  individual  or  institutional  accountability 
of  university  researchers  to  the  Technology  Area  Reviews 
and  Assessments  process,  the  allocation  of  funds  through 
peer-reviewed  grants  will  not  meet  all  the  needs  of  our 
defense  basic  research  program.  This  is  evidenced  by  the 
fact  that  from  FY97  to  FY02,  1 81  MURI  projects  have 
been  funded,  and  none  of  them  has  transitioned 
technology  to  the  warfighting  force.20  “* 

Multidisciplinary  University  Research  Initiative  (MURI) 
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.  Vern  Ehlers  (R-MI)  spoke  about  the  NSF  funding  level  in  H.R.  4818, 
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Clarification  of  Terminology 

Science  and  Technology  (S&T) 


Mused  tin  actions,  not  color  (>fJUiuiing. 

Technology  Readiness  Levels  (DoD  5000  Draft) 


S&T 


1,  Basic  principles  observed  and  reported,  paper  studies 

2,  Invention  begins  -  speculative  -  no  proof  or  detailed  analysis  to  support 
the  assumption 

3,  Research  and  development  is  initiated  -  studies  to  validate  predictions 

4,  Component  and/or  breadboard  validation 

5,  Component  and/or  breadboard  validation  with  reasonably  realistic 
supporting  elements 


6.  System/subsystem  model  or  prototype 

7.  Prototype  demonstration  near  or  at  planned  operational  system. 

8.  Actual  system  completed  and  "flight  qualified"  —represents  the  end  of 
true  system  development. 

9.  Actual  system  "flight  proven"  under  mission  conditions 
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Transition  is  recognized  between  all  TRL  levels. 


The  information  on  DoD  5000  rewrite  came  from  a  briefing,  the  title  page  of  which  is  given 
here. 


DoD  5000  Rewrite 

brief  for 

Deputy  Assistant  Secretary  of  the  Navy  (Air} 


Denis  Catalano 


Director,  NAVSEA  Acquisition  Support  Office 


Chairperson,  ASN{RD&A)  ABM  5000/ 
Deskbook  Working  Group 


7  Sept  £000 
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NAVY  S&T  REVITALIZATION 

“CONCLUSION” 

•  S&T  workforce  will  be  broken  within  the  current 
POM  and  incapable  of  pursuing  transformational 
S&T 

"The  Navy  has  lowered  its  level  of  intellectual  involvement  in  research 
and  development  and  weakened  its  entire  infrastructure,  which  at  the 
end  ofWWll  was  the  strongest  in  the  world.  For  a  service  that  sleeps 
on  its  weapons,  this  weakened  institutional  position  in  the  world  of 
science  and  engineering  is  dangerous \  .  .the  Navy  must  bring  the 
political,  military  and  civilian  scientist  cultures  into  a  mutually - 
supporting  relationship.  K* 

-  James  Colvard 

‘The  Bureaus  Did  Not  Go  On  Forever 
May  2002 ,  Naval  Institute  Proceedings 
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This  is  the  problem  in  a  nutshell.  This  is  what  we  must  declare  and  sell  internal  and  external  to 
the  Navy.  Kavetsky  -2002 
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